Multi touch ir drafting table

[1TONpete]

I am not sure if everyone is aware of what this is but I am using a wii mote as an ir sensor for a multi touch panel for the pc. This will be a mod to my drafting table. My drafting table has seen some years and is do for a new piece of acrylic sheet . This can also be done with a ps3 eye camera but requires some modification to the eye to make work. I have an extra eye but for now I will be using the wii mote for dev.
  A lil more detailed explanation of what I am doing:
I am basically fallowing this combined with Jon lee's wii mote stuff. I will eventually test ps3 eye to see which has better resolution.


Step 1 completed: gather stuff needed for multi touch: wiimote , bluetooth dongle , drivers and make an ir pen. Build a white board for testing. I did my first test of wii mote ir multi touch with my drafting table as a white board. Worked great, and had it installed and configured in minutes. Just no image projected on my drafting table, just a touch white board. I had an old remote control for a tv laying around I gutted it for its ir led to build a led pen.

I was waiting on money to buy a projector . I found a nice hd projector but it is not short throw so I will have to use a mirror. I have the money now and am still shopping for a nice hd projector. My plan is to mount the projector on the back of the drafting table facing away from it. So that the image has enough distance to travel to enlarge properly . I only have about 24" to work with. At 24" , I would only be able to get a 26" display. I want around 40+ inch display. So a reflector will be place in the back to reflect the projected image onto the surface of the acrylic table.

design image here

Now i can use multi touch with the pens as is right now. but I want to be able to use my fingers or hands etc , I need to turn that acrylic sheet into an ir sheet. I will fallow the steps used in the link pic above.

Here is where things get trick and fun. This is going to be a drafting table for graffix work in CS5 and 3d animation. I want a cool pen that can do a lot. I might need some help in doing what i want to do here.
   First I want to install a couple micro switches for r/l click for mouse. I might buy one of those mini ring finger mice , but for now I will be using a trac ball mouse with external wires for tact r/l clicks wired inside my pen. I installed 1 micro tact for led on/off for now, but my final pen will have 3 tacts, 1 thumb tact for led control and 2 for r/l click.
   The next part of my pen is the tips. I am not sure which way to go because I am not sure what is possible yet, but I have something cool planned here.
What I would like to see but I don't know if the resolution will let me. I want to put qr like codes in the tips of my pen. (the tips will be interchangeable btw)
The codes can correspond to brushes in photoshop etc. Can this be done?
    If that dont work i will just make different tips that have different shapes to them. like a fine point tip, a wide tip, a calligraphy tip , and maybe even a fiber optic brush.
    The next trick part i would like to see happen is install a rgb led inside the pen so that the tip can display the color I am using. I want to some how sample the color pallets in photoshop. Need help here also.


When all is said and done, I will need to build a second one on commission. My neighbor , who owns a couple art galleries here and NYC , stopped buy during white board dev, so I showed him some of the capabilities of the tech and he was sold. So I need to build this POC find out how much it will cost to build and convey that to him for a price for the job. The first drafting table will be for my own daily personal use.


Anyone with any input is welcome , and appreciated . If a coder gets involved I can provide compensation for code dev.

[Modded Matt]

this is very similar to the "smart boards" by pollyvision. The company I was formerly with was a distributor for pollyvision. We installed them in the coaches offices at the university of florida, they can draw their plans on what appears to be a white board, but the pen has no inc, the projector system knows what pen you have removed from the tray and knows to follow it and displays that color on the board.  the system records everything that is drawn/wrote and converts it to a digital format for distribution to the players.

very cool stuff. very expensive.

[1TONpete]

thanks Matt.
I actually know how to do that part, The projector system displaying the color of the pen I can do. I am hoping to get help from you guys on RGB led display on color pallet ? maybe a light sensor and a pic?

[Modded Matt]

I am not sure how you would detect the color. Like I said the smart boards I worked with had a chalk tray that the colored "pens" were chipped so the projector knew which color it was based on what was in the tray. As the rgb mixes light to create the different colors, I am unsure of how you would detect it.

[1TONpete]

I was thinking that in the coding of photoshop etc , there would be a color code for the pallets. the 2 primary selected ones. example 250 , 167 , 055 .
 My other idea was to take a sample like the paint matching systems do. I just touch the color pallet and sample the displayed color. Keep in mind the rgb led color does not have to be exact. It is just a reminder. I would like to see it automatic, by sampling the code for the pallets.
This is the pop up on pic color in Photoshop.

Here is the rgb code I want to grab and display in rgb led.

[frenulem - No.5417]

I am not sure how you would detect the color. Like I said the smart boards I worked with had a chalk tray that the colored "pens" were chipped so the projector knew which color it was based on what was in the tray. As the rgb mixes light to create the different colors, I am unsure of how you would detect it.

They have these in all schools over here, The way they done it here was that each pen was in it's own tray, and each tray had a LDR (Light dependent resistor) so when each pen was removed it then senses it, so like if some teacher lost their pen they would have a normal dry wipe one in there

I want to put qr like codes in the tips of my pen. (the tips will be interchangeable btw)
The codes can correspond to brushes in photoshop etc. Can this be done?

It'd be hard, their wouldn't be any way for the actual screen to read it, but what you could do is have each tip have a different resistance and then the chip see that resistance and then gets that brush type EG. a 47K res = a Wide tip, or 100K Res = Thin tip (If you use this beware of the resistor tolerance, like if you use a 100k resistor with a 10% tolerance could be read as either 90K or 110K)

Update later

[1TONpete]

Let me give you guys a different example of some the tech i have seen with this.
Reactables
http://youtu.be/vm_FzLya8y4
http://youtu.be/drRDMnizUQM

[toadzilla]

fren and matt i assume u guys are talking bout the smart tech smart boards we have em in every classroom here they are quite the machines to bad you couldn't replace the white board part with the touch based screen it would be so much cooler or a clear peicee in plac of the white board and a huge screen like feature behnd it

[1TONpete]

FREN-
It'd be hard, their wouldn't be any way for the actual screen to read it, but what you could do is have each tip have a different resistance and then the chip see that resistance and then gets that brush type EG. a 47K res = a Wide tip, or 100K Res = Thin tip (If you use this beware of the resistor tolerance, like if you use a 100k resistor with a 10% tolerance could be read as either 90K or 110K)

I like where you are heading with this. I posted examples above of how the qr type thing works with this tech. Maybe a tray system where if a tip is removed the tray measures the missing resistance.
Really as far as tips i can just make different physical tips and the ir sensors should see the difference. I like the idea of a fiber optic brush. Maybe use fishing line. Make a jig and glue a whole bunch of fishing line together to make a fiber optic brush. The big question is how to integrate all this tech with photoshop?
How about the RGB thing?

[1TONpete]

Today is the start of the actual build of this project. I have enough to get started. I am waiting on 100 IR leds.
Starting point pics.
This pic shows the projector , the table , trackball mouse , and the overlay for the acrylic sheet.

This is the overlay for the acrylic sheet. This stuff is a lay on frosted glass vinyl sheet .It does 2 things. 1 it helps with the projected image. 2 It has a bumpy surface for more accurate multi-touch compliance. 40$ at the home depot.

100$ of acrylic sheet.


The plan so far. I might not even use my drafting table , I might start over from scratch and build a new table. I have some nice oak I am thinking about using and staining really dark. You see the dimensions of the table are a nice 16:9 ratio. perfect for wide screen presentation. But i need storage bins on the right side of my display. also I had to buy my sheet in a 72" length. the table is only 48" wide. I would like to have a 48" of width of my display. that leaves me with 24" of space to put in a dock for my pen and have a couple storage areas. Thats perfect. Another problem I will run into if I use my drafting table is this design has a metal bar that goes across the middle of the viewing plain. Thats no good. So If I build to suit I might make something that fits my needs a lil better. I might actually put the new table on a pedestal and hide the projector underneath . Raise the table up off the floor some to give me more room for projection. Maybe no need for a mirror.
Todays plan design a new table and work on my ir led pen/mouse .

About pen mouse...
I need help figuring out how to implement analog slide pot into my pen for pressure control.  Like a real airbrush. I know how to install it and Photoshop will let me map pressure control to a keystroke, but how do I get increments or analog input on a keyboard. maybe a rotary wheel instead of pot. I would like to have a control system similar to an airbrush.

[hyper999]

- For your custom key stroke controls check out the teensy and teensyduino, it plugs in via usb, it can emulate a keyboard and you code for it in the arduino environment so it will be very easy to map your inputs to keystrokes.

- Pressure control using a rotary encoder or momentary on-off-on rocker/toggle switch will be much easier!

- RGB LED, check out this ambilight clone http://macetech.com/blog/node/113 and this one from adafruit http://www.ladyada.net/make/adalight/ perhaps you could modify one of these so it reads from the point on the screen where photoshop displays the current colour...

- Different tips, not sure the wiimote could detect different physical tips just based on their shape, I would use a dock for different tips with a micro switch or optical sensor (like the ones used to detect the disc in slot loading dvd drives) under each, then map these to keystrokes or whatever to change the tip in photoshop. If you could find suitable screen surface that was also conductive you could do like fren said and put a resistor in each tip, then have the controller measure the resistance between the wire to the pen and the screen surface...

[1TONpete]

What I have learned on the software side.
Photoshop and windows 7 have all sorts of presets and custom touch mapping capability . Photoshop more than 7.
Once a touch input is read by photoshop, I can setup key stokes etc. Photoshop does see the multi touch pen as a touch device.
A rotary encoder would be the east way to get an airbrush like control, but it wont be the same. maybe I can find a hid device that photoshop supports that has an analog input.

hardware side:
PC game controllers have analog sticks. The pc should be able to read a hid device with analog sticks. I want to test to see if photoshop will see a game controller.
i will be making a dock for my tips. the tips will be mapped to keystrokes. The fiber optic brush  can be used as a real brush I found info on that at least.the "blobs" that are produced from the brush can be translated into bristles . On this same topic I can also finger paint with multiple points but I need to look more into that plugin.
I designed a new cabinet this one is larger than my drafting table. I have the spare materials and am going to build the cabinet out of oak. 
LED rgb is still up in the air. its not 100% needed so its not my main focus.
I would like great controls over cool looks.
I did find out my motion joy software for connecting dualshock 3 controllers to pc , has custom mapping also... might be able to adapt a dualshock 3 and use the pressure sensitivity on triggers or buttons or even map analog sticks to something else.
I am sick as a dog right now so I won't be updating for a couple days.

[1TONpete]

update bump:
led jig. 1.5 forward v ir leds x 8 . @ 7/8" apart. Total 12 strips. making 96 leds total. 12v dc power each strip ran parallel to each other. This 12v will come from pc 12v rail.



All my cabinet pieces pre-cut ready for assembly .



update: 2/28/12
Got the cabinet together, after a long morning, but that's another tale.
Mounted the mech for angle adjustment on my table top. It is from a window opener.
first up new tools...
18v cordless drill and saw.



this thing is huge....






http://youtu.be/4vUSaqYbZw8