IllumiPaper Applications
The IllumiPaper research platform (E) provides paper-integrated visual feedback without losing the sensory richness and flexibility of paper (A) and supports several applications including educational grid puzzles (B), interactive tool palettes (C) or even math exercise sheets (D).

Publications

  • Illuminated Interactive Paper with Multiple Input Modalities for Form Filling Applications

    Illuminated Interactive Paper with Multiple Input Modalities for Form Filling Applications.

    Klamka, K.; Büschel, W.; Dachselt, R.;

    In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces.ISS '17, Brighton, United Kingdom.ACM,434-437,2017.10.1145/3132272.3132287

    @inproceedings{Klamka2017d_IllumiPaper_ISS,
       author = {Konstantin Klamka and Wolfgang B\"{u}schel and Raimund Dachselt},
       title = {Illuminated Interactive Paper with Multiple Input Modalities for Form Filling Applications},
       booktitle = {Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces},
       series = {ISS '17},
       year = {2017},
       month = {10},
       isbn = {978-1-4503-4691-7},
       location = {Brighton, United Kingdom},
       pages = {434--437},
       numpages = {4},
       doi = {10.1145/3132272.3132287},
       url = {https://doi.org/10.1145/3132272.3132287},
       acmid = {3132287},
       publisher = {ACM},
       address = {New York, NY, USA},
       keywords = {Digital pen and paper, electroluminescence, augmented paper, visual feedback, Anoto, form filling, printed electronics}
    }

    List of additional material

    Video, Project page

  • @inproceedings{klamka2017c,
       author = {Konstantin Klamka and Raimund Dachselt},
       title = {IllumiPaper: Printed Displays for Novel Digital Pen-and-Paper User Interfaces},
       booktitle = {Mensch und Computer 2017 - Workshopband},
       year = {2017},
       month = {9},
       location = {Regensburg},
       pages = {625--628},
       numpages = {4},
       doi = {10.18420/muc2017-demo-0289},
       url = { https://doi.org/10.18420/muc2017-demo-0289},
       publisher = {Gesellschaft f\"{u}r Informatik e.V.},
       address = {Regensburg},
       keywords = {digital pen and paper, electroluminescence, visual feedback, Anoto, augmented paper}
    }

    List of additional material

  • IllumiPaper: Illuminated Interactive Paper

    IllumiPaper: Illuminated Interactive Paper.

    Klamka, K.; Dachselt, R.;

    In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.CHI '17, Denver, CO, USA.ACM,5605-5618,2017.10.1145/3025453.3025525

    @inproceedings{klamka2017a,
       author = {Konstantin Klamka and Raimund Dachselt},
       title = {IllumiPaper: Illuminated Interactive Paper},
       booktitle = {Proceedings of the SIGCHI Conference on Human Factors in Computing Systems},
       year = {2017},
       month = {5},
       location = {Denver, CO, USA.},
       pages = {5605--5618},
       numpages = {14},
       doi = {10.1145/3025453.3025525},
       url = {http://dx.doi.org/10.1145/3025453.3025525},
       publisher = {ACM},
       keywords = {Digital pen and paper, electro-luminescence, pen interaction, visual feedback, Anoto, thin-film display, augmented paper}
    }

    List of additional material

    Video, Talk

  • Demonstrating IllumiPaper: Illuminated Interactive Paper

    Demonstrating IllumiPaper: Illuminated Interactive Paper.

    Klamka, K.; Dachselt, R.;

    In Accompanying CHI 2017 Interactivity Research Demo of the SIGCHI Conference on Human Factors in Computing Systems.CHI '17, Denver, CO, USA.ACM,2017.

    @unpublished{klamka2017b,
       author = {Konstantin Klamka and Raimund Dachselt},
       title = {Demonstrating IllumiPaper: Illuminated Interactive Paper},
       booktitle = {Accompanying CHI 2017 Interactivity Research Demo of the SIGCHI Conference on Human Factors in Computing Systems},
       year = {2017},
       month = {5},
       location = {Denver, CO, USA.},
       publisher = {ACM},
       keywords = {Digital pen and paper, electro-luminescence, pen interaction, visual feedback, Anoto, thin-film display, augmented paper}
    }

    List of additional material

    Video Preview, Video

 

Table of Contents

This research project website provides additional information about the fabrication of our IllumiPaper system including the four main components of the digital stylus, our augmented papers, the smart clip and its charging module as well as the Android device. In the following, we provide a short table of contents with skip links to easily jump to all description parts:

Abstract

Due to their simplicity and flexibility, digital pen-and-paper solutions have a promising potential to become a part of our daily work. Unfortunately, they lack dynamic visual feedback and thereby restrain advanced digital functionalities. In this paper, we investigate new forms of paper-integrated feedback, which build on emerging paper-based electronics and novel thin-film display technologies. Our approach focuses on illuminated elements, which are seamlessly integrated into standard paper. For that, we introduce an extended design space for paper-integrated illuminations. As a major contribution, we present a systematic feedback repertoire for real-world applications including feedback components for innovative paper interaction tasks in five categories. Furthermore, we contribute a fully-functional research platform including a paper-controller, digital pen and illuminated, digitally controlled papers that demonstrate the feasibility of our techniques. Finally, we report on six interviews, where experts rated our approach as intuitive and very usable for various applications, in particular educational ones.

Video


Full video of IllumiPaper.

Full video of the Preview for IllumiPaper.

Extended Technical Schema

IllumiPaper Extended Technical Schema

Technically, our setup consists of four components: A digital stylus (S), different augmented papers (P), a smart controller clip (C) and an Android device (D).

SDigital Stylus

IllumiPaper Pen Schematic

We used the Anoto-based digital pen Maxell DP-201 for our project. However, it does not matter which model or sensor technology is used as along as the pen stream its position and state data.

We stream the following data values:

  • unique pen identifier
  • x and y position
  • pressure
  • event type
  • timestamp

Data Streaming

In order to stream the pen data, we built a thin application that receive stroke and state data via Bluetooth from the pen and forwards the data in the Open Sound Control (OSC) format via WiFi to our mobile Android application.

Components*

Hardware for our streaming solution
Component Model
Notebook HP Compaq tc4400
Bluetooth Adapter Ednet Bluetooth USB V2.0 C2 Adapter
Digital Stylus Maxell Penit DP-201 Digital Pen
Local WiFi Apple AirPort Express Base Station

*Please note: In order to save costs, we used hardware that exist in our research lab. Of course, you can use newer models and components, however, this is not necessary.

Setup

IllumiPaper Anoto PenIllumiPaper Pen SetupIllumiPaper OSC Bridge

Please note: Since Anoto™ introduced the commercial Android Live SDK, pens can be paired directly, which enable even faster mobile setups.

PAugmented Papers

IllumiPaper Augmented Paper Schematic

Paper Traces

Fabrication Methods for Traces
Method Conductive Material Prototyping Level
A: Copper Tape Copper Foil Tape with Conductive Adhesive – 6mm easy, cheap and fast
B: Conductive Ink Pen Circuit Scribe Pen easy, cheap and fast
C: Conductive Inkjet-Printing Mitsubishi Imaging (MPM) Silver Nanoparticle Ink advanced, higher initial setup costs
  • Method: Copper tape traces

We used self-adhesive copper tape to prototype high-conductive traces by sticking this tapes on standard paper. However, the self-adhesive copper tape looks a little bit cumbersome at some points, it is a good starting point to initially test a IllumiPaper design.

IllumiPaper Copper Tape

 

  • Method: Conductive ink drawn traces

Conductive ink pens (e.g., with circuitscribe or Bare Conductive Ink) enable instant circuit sketching. In addition, conductive ink pens can be combined with other types of trace fabrication (e.g., copper tapes or printed traces). We use conductive z-axis tape to bridge the types or to connect a component or module.

IllumiPaper Ink Drawn

 

  • Method: Conductive inkjet-printed traces

In addition, traces can be directly inkjet-printed on paper with conductive inks. These advanced fabrication method can be used to increase the degree of quality and integration of IllumiPapers.

Epson ET-2500 | Conductive Inkjet Printer

We used a Epson ET-2500 for the conductive ink printing. The tubes, ink tanks and the printer head of the Epson work well since the parts are clean and empty (many thanks to the HCI group in Saarbrücken for the help). With the Brother MFC-480DW, we have bad experiences since the printers already come, to our surprise, with pre-charged ink (printer head, printer tubes). This would contaminate the conductive ink.

conductive inkjet printing

Please make sure that you respect necessary safety instructions of the silver-nanoparticle ink.

Our smart-clip with a printed inkjet-printing sheet. The conductive traces and electrodes serve as a basis for screen printed EL substrates.

IllumiPaper Ink Printed

Paper Connectors

 

IllumiPaper
Image Part Description
IllumiPaper Conductive Z-tape 3M Z-Axis Conductive Tape 9703 – 2″x6″ (50mm x 150mm) Strip
IllumiPaper Self-adhesive Copper Tape Self-adhesive Copper Tape We used self-adhesive copper tape to prototype high-conductive traces by sticking this tapes on standard paper. However, the self-adhesive copper tape looks a little bit cumbersome at some points, it is a good starting point to initially test a IllumiPaper design. Further advanced fabrication methods such as conductive inkjet printing can be the next step to achieve a better integration.
IllumiPaper Connector Experimental Breakout 0.1″ alligator clip breakouts

Paper Displays

IllumiPaper
Image Part Description
IllumiPaper Multi-Contact EL Foils Cuttable multi-contact electroluminescent foil with prefabricated contacts We used multi-contact electroluminescent foils with prefabricated contacts in different colours (e.g., red, blue, white) that can be easily cut and/or covered with printed self-adhesive transparent foil. This allows us to test several illuminations in a fast way.
IllumiPaper Custom Contact EL Foils Cuttable multi-contact electroluminescent foil with customizable contacts
IllumiPaper flexible EL Foil ELastoLite Panel – 5×3 inches – Orange
and
ELastoLite INV133 Inverter
In order to investigate the flexibility of electroluminescent displays, we tested paper-attached ELastoLite panels that are very robust and crushable.
Screen printed electroluminescent display We experimented with screen printed electroluminescent displays that allow paper displays on the paper itself.

Besides the related work that is mentioned in detail in our paper (e.g., PrintScreen), there is a great Electroluminescent Screen Video Tutorial from the Zurich University of the Arts (ZHdK) (Interaction Design Program) that shows the fabrication process in detail.

Ynvisible Electrochromic Seven-segment Display We used flexible transparent electrochromic e-paper like displays from Ynvisible that can be also printed directly on paper. Electrochromic materials change their color when electricity is applied. In our case, the seven-segment display gets blue.
Printed Self-adhesive Transparent Foil We used printed self-adhesive transparent foils to cover the electroluminescent foils. In transparent regions the electroluminescent panel shines through, while the black-printed areas work as stencils and block the illumination. This allows us to test several illuminations in a fast and easy way. The foils are printed in our local copyshop.

EL Screen Printing

Smart Clip

 

Tool List

Part List

IllumiPaper
Image Part Description
SparkFun Power Cell – LiPo Charger/Booster PRT-11231 We used a power management board from Sparkfun with a Microship MCP73831 chip. This allows us, versatile charging capabilities including micro-usb and inductive charging. In order to avoid any damage on the micro-usb port on the board during the development (cf. Sparkfun customer reviews), we strengthen this part with a custom-printed part.
LiPo Accu – 400mAh PRT-10718 We used a standard 400mAh lithium polymer battery to power the smart clip to power the smart clip. The battery provides electric power to both: to the microprocessor including its associated logic parts and the high-voltage frequency AC circuit that is generated by our EL inverter for the EL illumination.
RFduino For our smart controller clip, we used the RFduino microcontroller which basically consists of a Nordic Semiconductor nRF51822 chip with a Arduino-compatible bootloader. For rapid prototyping, we decided to use the bigger version with 0.1″ pin breakouts. However, for further miniaturizations a smaller SMD version of the microcontroller (size of a finger-tip) is still available.
FTDI Programmer In order to upload new firmware to our research platform, we used this tiny FTDI programmer that can be directly stacked RFduino. However, every other 3.3V FTPI programmer works as well. You have simply to hook up the following pins: receive pin (RX, 0), transmit pin (TX, 1), reset pin, ground and +3V.
EL-Inverter Conrad WE-50: 5 V/DC black We used a classic 5V EL-Inverter that produces 110 V/AC with a frequency of 700 Hz at output. This inverter can illuminate an overall surface area up to 105 cm².
PhotoMOS-Relay, Panasonic AQY210EH Optoisolators are often too slow to turn off at the very brief zero crossings when it is fed with ~1kHz AC current. Therefore, we use a more appropriate OptoMOS SSR switching elements to control the EL segments and handle the brief zero crossings.
Adafruit 74HC595 Shift Register To extend the limited GPIO pins of our microcontroller, we used shift register to extend the addressable digital channels, which we need, e.g., for our EL channels. Therefore, we used the popular Adafruit 74HC595 shift register, which are basically Texas Instrument 74HC595 Shift Register.
Prototyping PCB Phenolic paper, 0.1″ (2.54mm), 98003C12 We used these prototyping boards as a basis for our circuits.
80 mm x 52 mm 35 µm
Contact spacing 2.54 mm
Pololu 3.3V, 600mA Step-Down Voltage Regulator D24V6F5 Since our EL-Inverter requires 5V and our microcontroller 3.3V, we used a step-down voltage regulator to provide power to both from our power management board.
Adafruit 12-Key Capacitive Touch Sensor Breakout – MPR121 We used the MPR121 chip (hooked up via I2C) to handle up to 12 individual touch pads.
ADS1115 16-Bit ADC – 4 Channel with Programmable Gain Amplifier We integrated the ADS1115 chip (hooked up via I2C) to extend up to 4 single-ended high-precision input channels. We used the additional ADC channels to sense our resistive paper flex gestures.
SPDT Switch Simple SPDT switches serve us as on/off switches for our prototype. Therefore, we used one for our power management board and one for the battery to ensure extra-security while working on the open prototype.
Adafruit Inductive Charging Set – 5V @ 500mA max We aim to support versatile charging capabilities and therefore integrate a inductive wireless charging module, which allows wireless charging.
Mill-Max Pogo Pins: Single Row Series 819 (815-22-010) To avoid loose contacts in our clip mechanism caused by surface irregularities or lateral shifts, we use small spring-loaded pogo pins. These tiny pogo pins compensate any contact problems.
Makeblock Socket Cap Screw M4x8-Button Head We used M4 Cap Screws with metric threads to fix our circuit boards to our custom-printed case parts.
Makeblock Plastic Spacer 4x7x2 These tiny plastic spacer serve us to ensure minimal distances between our stacked circuit boards and thereby avoid bypasses.
M1 screws and nuts These very tiny screws are used to fix the bulldog with our prototyping PCB (the screws fit in the holes of the PCB).
Jumper wire Yv,
1 x 0.20 mm²,
Black
We used isolated standard jumper wire to hook up all sensors and output channels. However, there are not special requirements for the wires. Basically, you can use nearly every isolated cable.
Foam Rubber Sheet,
2.0 mm x 200 mm x 300 mm, black
We used pieces of foam rubber to ensure a secure hold of the paper in our smart clip.
Standard 0.1″ (2.54mm) Pin Headers We used several standard pin headers to design our smart prototype clip as flexible as possible without hardwiring all components. For example we built stackable shields with perfboards and pin headers to extend our research platform modular.
Standard 0.1″ (2.54mm) Crimp Connectors In order to create custom cables for our research platform (e.g., flexible stackable EL channel pin layout), we used standard 0.1″ (2.54mm) crimp connectors.
Bulldog Clip,
metal,
50 mm
We used this 50 mm bulldog clip for our smart clip. Therefore, we drilled holes with our rotary tool in the sides and mounted the prototyping PCBs with M1 screws and nuts to it.
Diverse boards including fibreboards, perfboards and plastic foils
wood-pulp board, Gerstaecker 14758
PP3DP TierTime UP! ABS white 1,75mm 700g Filament for UP! We used white ABS from PP3DP to print out all case parts for our IllumiPaper research platform. Of course, you can use any other filament, such as PLA, to fabricate the parts. However, you have to pay attention that the filament is supported by your 3D printer.
Angled Micro USB Cable The angled micro usb cable fits perfectly into our 3D printed case and is used to charge our smart clip via USB.
USB extension cable Since it is very tricky and annoying to plug the FTDI programmer directly into the computer or notebook, we used a USB extension cable (type A) to comfortably connect the FTDI programmer to upload new software updates on the micro-controller.

Microcontroller, Shift Register and PhotoMOS-Relays

Note: Optoisolators are often too slow to turn off at the very brief zero crossings when it is fed with ~1kHz AC current. Therefore, we use a more appropriate OptoMOS SSR switching element (see PhotoMOS-Relay, Panasonic AQY210EH).

Charging Circuit & EL-Inverter

Circuit, back

Note: In order to miniaturize the layout a custom printed circuit can save a lot space. We used for our circuit design rapid prototyping methods, which are cheap, easy (soldering + standard cables) and are available for everyone.

Inductive Charging

Case

We 3D printed basically three parts for our Smart Clip.

You can download the STL-Files here:

 

Charging Module

Tool List

Part List

Inductive Charging Module
Image Part Description
Adafruit Inductive Charging Set – 5V @ 500mA max We aim to support versatile charging capabilities and therefore integrate a inductive wireless charging module, which allows wireless charging.
Makeblock Socket Cap Screw M4x8-Button Head We used M4 Cap Screws with metric threads to fix our circuit boards to our custom-printed case parts.
Jumper wire Yv,
1 x 0.20 mm²,
Black
We used isolated standard jumper wire to hook up all sensors and output channels. However, there are not special requirements for the wires. Basically, you can use nearly every isolated cable.
Foam Rubber Sheet,
2.0 mm x 200 mm x 300 mm, black
We used pieces of foam rubber to fix all components in the charging module case.
USB Micro-B Breakout Board This tiny micro-usb breakout board allow us to seamlessly connect our inductive charging module. Our 3D printed case is designed that the board fits perfectly in.
Acrylic glass GS, transparent,
fluorescent blue
This acrylic glass plate is used as a bottom plate for our inductive charging module. This blue shimmering plate allows to see the electronic and components inside the module. However, it is of course possible to use any other material such as normal acrylic glass, wood or metal.
Angled Micro USB Cable The angled micro usb cable that we already used to charge our smart clip via USB could also be used for our inductive charging module.
PP3DP TierTime UP! ABS white 1,75mm 700g Filament for UP! We used white ABS from PP3DP to print out all case parts for our IllumiPaper research platform. Of course, you can use any other filament, such as PLA, to fabricate the parts. However, you have to pay attention that the filament is supported by your 3D printer.

Case

You can download the STL-Files here:

 

Android Device

 

We used Vatavu et al.’s $P Point-Cloud Recognizer to process character- and glyph-based input.

Application Screens

Our mobile Android application basically consists of three parts, which are implemented as tabs:

  • An Application Tab (see figure 1+2)
  • A Stylus Tab (see figure 3)
  • A Controller Tab (see figure 4+5)

Our work was in part funded by the German Federal Ministry of Education and Research (Twenty20 — Partnership for Innovation, project fast, grant no. 03ZZ0514C).