Magic Wand: A Hand-Drawn Gesture Input Device in 3-D Space with Inertial Sensors

Pages 6
Views 20
of 6
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Magc Wad: A Had-Draw Gesture Iput Devce 3-D Space wth Iertal Sesors Sug-Jug Cho, Jog Koo Oh, Wo-Chul Bag, Wook Chag, Euseok Cho, Yag Jg, Jookee Cho, Dog Yoo Km {sug-ug.cho, og.oh,, wook.chag, euseok.cho,
Magc Wad: A Had-Draw Gesture Iput Devce 3-D Space wth Iertal Sesors Sug-Jug Cho, Jog Koo Oh, Wo-Chul Bag, Wook Chag, Euseok Cho, Yag Jg, Jookee Cho, Dog Yoo Km {sug-ug.cho, og.oh,, wook.chag, euseok.cho, g.yag, hadle.cho, Samsug Advaced Isttute of echology PO Box, Suwo, , Korea Astract hs paper presets a gesture put devce, Magc Wad, wth whch a user ca put gestures 3-D space. Iertal sesors emedded t geerate accelerato ad agular velocty sgals accordg to a user's had movemet. A traectory estmato algorthm s employed to covert them to a traectory o 2-D plae. he recogto algorthm ased o Bayesa etworks fds the gesture model wth the maxmum lkelhood from t. he recogto performace of the proposed system s qute promsg; the wrter-depedet recogto rate was 99.2% o average for the dataase of 5 wrters ad 3 gesture classes. Keywords: Gesture put devce, Hadwrtg recogto, Bayesa etworks, raectory estmato, Iertal avgato system, Accelerometer, Gyroscope. Itroducto As the uqutous computg evromet ecomes wdespread these days, the role of computers has chaged from massve computg devces to assstat devces of our daly lves. Accordgly, more atural teracto methods eyod tradto keyoards ad mouse have ee studed. Speech recogto [], vso-ased gesture recogto [2] ad o-le hadwrtg recogto [3] are popular examples. Amog them, the ole hadwrtg put method, whch trascres huma had movemets to characters ad gestures, has the advatage of atural ad portale teracto. It s very atural ecause people have ee accustomed to usg pes ad papers sce chldhood. Also a small wrtg surface of a few ches s eough for applyg t. Covetoal o-le hadwrtg recogto systems am at recogzg traectores wrtte o 2-D wrtg surfaces. he wrtg surface s ealed usually y talet types of devces such as opaque talets, touch pads, talet PCs ad we pads. Whe people wrte characters ad gestures y these devces, pe postos o 2-D plae are dgtzed y sesg pressures [4] or electro-magetc sgals of talets [5]. hese devces have the advatage of hgh resoluto ad hgh samplg rate dgtzato. However, they have the lmtato that people should wrte oly o talets. I order to exted the wrtg area eyod talets, ew types of sesors are employed such as optcal sesors ad ultrasoc waves. I the case of optcal sesors, a camera mouted o the pe tp captures the mage aroud t. he the coordate of the pe tp s calculated y aalyzg uque mage patters [6] or comparg chages etwee cosecutve put mages [7]. I the case of ultrasoc waves, a pe emts ultrasoc waves ad recevers eghorhood compute dstaces from t [8]. Due to these sesors, wrtg surfaces are exteded to ay plat surfaces. By employg ertal sesors, the wrtg area s further exteded to 3-D space. Iertal sesors measure the erta of oects; accelerometers measure acceleratos ad gyroscopes measure agular veloctes. hey are small eough to e emedded hadheld formato devces. Moreover, they do ot requre ay exteral referece devces. herefore, users ca put gestures almost ay place. hs s a g advatage over the approach of employg exteral sesors such as ultrasoc recevers/emtters [9] ad cameras [0], whch requre fxed stallato space. However, they have the lmtato that traectores are ot measured drectly ut calculated from sesor sgals so that roust sgal processg techques are ecessary. hs paper presets a gesture put devce, Magc Wad, that recogzes traectores of had movemets 3-D space. Whe a user wrtes gestures 3-D space wth the wad, ts ertal sesors, 3-axs accelerometers ad 3- axs gyroscopes, covert had movemets to accelerato ad agular velocty sgals. he, a traectory estmato algorthm coverts them to traectores. Fally, a recogto algorthm matches the traectores wth Bayesa etwork-ased gesture models. 2. System overvew 2. System usage I order to show the applcalty of a 3-D put devce commercal products, we have made a prototype remote cotroller wth oly oe utto. ypcal remote cotrollers have tes of uttos. Dfferet commads are Proceedgs of the 9th It l Workshop o Froters Hadwrtg Recogto (IWFHR /04 $ IEEE mapped to dfferet uttos so that users cotrol electroc applaces such as V s ad DVD players y pressg uttos. May customers compla the dffculty to fd the proper utto amog so may small uttos. However, the proposed system has oly oe utto whch actvates sesor sgals. Users cotrol applaces y drawg gestures whle pressg the actvato utto. Fg. shows the pcture of the proposed system. It looks otcealy smple ad slm compared to covetoal large remote cotrollers wth a lot of uttos. he typcal scearo of ts usage s as follows. Whe a user wats to ssue a commad, he draws the gesture shape correspodg to t whle pressg the actvato utto. After the utto s released, the gesture shape s recogzed. he, proper IR cotrol codes are fetched ad trasmtted to a V va ts IR LED. By usg the tradtoal IR codes, covetoal Vs are cotrolled y gestures wthout ay modfcato to them. Fg. : Magc Wad: the proposed gesture put devce as a form of a remote cotroller 2.2 Hardware compoets he Magc Wad cossts of accelerometers ( Zaxs, ad Y-axs chps, gyroscopes ( -, Y-, Z- axs chps, a aalog-to-dgtal coverter (ADC, a dgtal sgal processor (DSP, a flash memory, a frared (IR LED, a seral port terface ad a lthum-ro attery (Fg. 2. Accelerometers measure acceleratos ad gyroscopes measure agular velocty. he measured sgals are coverted to dgtal sgals y ADC. DSP rus sgal processg ad recogto algorthms. Flash memory stores program codes ad data. A seral port terface s used for trasmttg sesor data to PC whe collectg data ad trag gesture recogzers. IR Y Z,Z Y Battery Gyroscope Accelerometer Fg. 2: Hardware compoets of Magc Wad DSP Mem ADC Seral port 2.3 Software compoets he gesture recogto system has the compoets of a traectory estmato algorthm ad a gesture recogto algorthm. he traectory estmato algorthm gets the accelerato ad agular velocty sgals from sesors, ad coverts them to a had-movemet traectory. he gesture recogto algorthm gets the traectory ad Gesture put Gesture class Iertal sesors Gesture recogto Accelerato Agular velocty raectory o 2-D plae Fg 3: Software compoets of Magc Wad raectory estmato classfes t to oe of predefed gesture classes. Fg. 3 shows the overall software compoets ad data flow. We choose to classfy traectores stead of raw sgals ecause of two reasos. Frst, sgal varatos ca e greatly reduced the traectory doma compared to the raw sgal doma. Iertal sesor sgals are sestve to the varato of moto status ad wrters. Eve a smple traectory ca e made y dfferet ways of movemets; some people ted to wrte slowly ad other fast. Also the posture of the devce, how t s oreted the 3-D space, affects sesor sgals. I traectory doma, these moto ad posture varatos are removed. Secod, traectory doma, we ca apply tradtoal o-le hadwrtg recogto algorthms. hey have ee studed for decades so that relale recogto performace s expected, provded that traectores ca e staly estmated. 3. raectory estmato algorthm he traectory estmato algorthm coverts raw sesor sgals to traectores 3-D space ad fally proects t oto 2-D plae [-3]. he frst step to realze the proposed system s to detfy physcal moto propertes,.e., three dmesoal posto ad oretato formato, whch falls to category of moto trackg. here are varous kds of moto trackg algorthms avalale ad the proposed system utlzes the moto trackg method wth ertal sesg techologes. Wth three axs accelerato ad agular rate measuremets, the theory of ertal avgato system (IS theoretcally guaratees the posslty of computg posto ad oretato formato of a oect movg the 3-D space []. o apply a IS theory, we defed the coordate system of the ody coordate ( ad the avgato coordate ( 3D space, as show Fg. 4. he avgato coordate ( s the statg pot of traectory exteral to the put devce., Y, ad Z axes are perpedcular to oe aother, where the drecto of Z s parallel to the drecto of earth gravty (g. It does ot chage eve whe the devce s moto. he ody coordate ( s fxed o the devce. It s alged wth the axes of the ertal sesor chps (IMU: ertal measuremet ut., Y, ad Z axes are perpedcular Proceedgs of the 9th It l Workshop o Froters Hadwrtg Recogto (IWFHR /04 $ IEEE to each other, where the drecto of Z axs s alged wth the core axs of the devce. herefore, t s chaged accordg to the moto of the devce ody. Fg. 4: Coordate system: the ody coordate ( ad the avgato coordate ( Whle a user s drawg a gesture, the IMU measures A = A A A ad agular rate the accelerato [ ] x y z [ ω ω ω ] ω = of each axs the ody coordate G x y Y z Z G V, P Y avgato coordate ( Earth ravty (g Body coordate ( (. he, usg A ad ω, the accelerato A [ ] = Ax Ay Az the avgato coordate ( s calculated y the state-space equato [] of the devce. By tegratg A twce, we ota the hadwrtte traectory P [ ] = Px Py P the avgato z coordate (. he goverg equatos of moto trackg used ths paper are expressed as follows: P& = V V& = C A G & θ = ω y cosφ ω z sφ ω y sφ + ω z cosφ ( ψ& = cosθ & φ = ω x + ( ω y sφ + ωz cosφ taθ where the suscrpt deotes the avgato coordate, ad deotes the ody coordate, V s the rate of chage of posto,.e., velocty show the avgato coordate, G s the costat gravty vector show the avgato coordate, ( ω x, ωy, ωz s the ertal agular rate vector show the ody coordate, ( φ, θ, ψ = (roll, ptch, yaw are Euler Magc Wad agles. Here, the matrx C = C refers to the drecto cose matrx descrg the rotato relatoshp etwee the avgato coordate ad ody coordate ad s the fucto of Euler agles as show Eq. (2. However, the aove descred algorthm ca ot e drectly appled to the proposed system sce the IS leads to a uouded growth of error due to may tegrato steps volved. A typcal IS uses perodc or aperodc resettg procedure to remove the error growth, whch s ot a feasle soluto for the small ad low-cost systems cludg the proposed system. Fortuately, the soluto of ths prolem has ee detaled ad solved [2] ad s called zero velocty compesato (ZVC. After recostructg 3D moto formato, we proect the recovered 3D traectory to a 2D wrtg plae y fdg the optmal wrtg plae the sese of mmum dstorto to the orgal pot postos [2]. he purpose of ths process s to reduce the wrtg plae varato of the estmated traectores. 4. Gesture recogto algorthm We apply the o-le hadwrtg recogzer ased o Bayesa etworks [4-5] for recogzg traectores estmated from ertal sesor sgals. he recogzer models depedeces etwee pots ad asc strokes explctly. It showed favoraly comparale recogto rates to covetoal approaches ased o template matchg method ad hdde Markov models recogzg dgts ad Korea Hagul characters [4-5]. 4. Itroducto to Bayesa etworks A Bayesa etwork s a drected acyclc graph (DAG whose odes represet radom varales ad whose arcs relatoshps etwee them [6]. It effcetly ecodes the ot proalty dstruto of a large set of radom varales. Whe a Bayesa etwork S has varales:, 2, L, ad pa( deotes the radom varales from whch depedecy arcs come to, the ot proalty of, 2, L, s gve as follows:, 2, L, = pa(. (3 = I ths paper, the codtoal proalty s represeted y the codtoal Gaussa proalty [4]. Whe a multvarate radom varale depeds o,, L, the codtoal proalty dstruto s gve as follows: cos( ψ cos( θ s( ψ cos( θ s( θ C = s( ψ cos( φ + cos( ψ s( θ s( φ cos( ψ cos( φ + s( ψ s( θ s( φ cos( θ s( φ (2 s( ψ s( θ + cos( ψ s( θ cos( φ cos( ψ s( φ + s( ψ s( θ s( φ cos( θ cos( φ Proceedgs of the 9th It l Workshop o Froters Hadwrtg Recogto (IWFHR /04 $ IEEE = x = x, L, = x. (4 d / 2 / 2 = (2π Σ exp[ [ x u] Σ [ x u]] 2 he mea µ s determed from the lear weght sum of depedat varale values Z = [ x, L, x,] as follows: u = WZ. (5 where W s a d k lear regresso matrx, d ad k are the dmeso of ad Z respectvely. 4.2 Gesture model A gesture s represeted herarchcally y modelg ts compoets ad relatoshps amog the compoets [4]. I the frst level, a gesture model s composed of asc stroke models ad ther relatoshps. I ths paper, a asc stroke deotes a early straght trace whose gloal drecto s dfferet from those of coected traces wrtg order. I the secod level, a asc stroke model s composed of pot models ad ther relatoshps. Fally, a pot s modeled wth 2-D Gaussa dstruto for ts -Y posto. A pot model s represeted y a 2-D Gaussa dstruto for (x, y coordates of ts correspodg pots 2-D plae. It correspods to a sgle ode Bayesa etworks. A asc stroke model s composed of pot models ad ther relatoshps, called as WSRs (wth-stroke relatoshps. It s costructed y recursvely addg md pot models ad specfyg WSRs. A md pot s the pot at whch the legths of the left ad the rght partal strokes are equal. A WSR s represeted as the depedecy of a md pot from two ed pots of a stroke. Fg. 5 shows the recursve costructo example of a asc stroke model. Fg. 5 (a shows a example of asc stroke staces. At the frst recurso ( d =, IP s added for modelg p 's wth the WSR from EP 0 ad EP (Fg. 5 (. At d = 2, IP 2 ad IP 3 are added for the left ad the rght partal asc strokes (Fg. 5 (c. hs recurso stops whe the covaraces of ewly added pot models ecome smaller tha a predetermed threshold. p p 2 3 p ep 0 ep ep 0 p 2 p OˆP p 3 ep EP0 EP IP O P EP0 EP IP 2 IP OŠP Fg. 5: Example of recursve costructo of a asc stroke model A gesture model s costructed y cocateatg asc stroke models accordg to ther wrtg order ad IP 3 specfyg ter-stroke relatoshps (ISRs. ISRs are represeted y depedeces amog asc stroke ed pots. Fg. 6 shows a Bayesa etwork ased gesture model wth strokes ad the stroke recurso depth d = 2. EP 's are the stroke ed pot models ad IP, 's are the teral pot models of the -th asc stroke. he rght ed pot of the prevous asc stroke s shared wth the left oe of the followg asc stroke. ISRs are represeted y the arcs etwee EP 's, ad WSRs are represeted y the comg arcs to IP, 's. EP 0 IP,2 IP, IP,3 EP IP 2,2 IP 2, IP 2,3 EP 2 UUU UUU EP IP,2 IP, EP Fg. 6: Gesture model wth asc strokes ad the stroke recurso depth of Matchg algorthm Each gesture class m has a correspodg gesture model λ m [4]. A gesture put, a traectory pot sequece of O, L, O, s recogzed y fdg the gesture model λ * whch produces the hghest model lkelhood as follows: * λ = arg max m λm O, L, O. (6 = arg max m λm O, L, O λm he model lkelhood s calculated y matchg stroke teral pot models ( IP s ad stroke ed pot models ( EP s of gesture models (Fg. 6 wth the put pot sequece. Because oudares of asc strokes are ot explctly specfed the put pot sequece, all the possle asc stroke segmetatos should e searched. After a gesture put s segmeted to asc strokes, asc stroke ed pots are matched to EP s. he each asc stroke s recursvely resampled to md pots ad matched to IP s. Whe a gesture model G wth asc stroke models matches the put pots O, L, O, ad oe asc stroke segmetato stace s deoted as γ = ( t0, t, L, t, t0 =, L, t = ad the whole set as Γ, the model lkelhood s calculated as follows: O, L, O λm (7 = EP = O( t O( t, L, O( t γ Γ = 0 = d 2 k= IP, k = p, k 0 ( O( t, t pa( IP, k IP,3 Proceedgs of the 9th It l Workshop o Froters Hadwrtg Recogto (IWFHR /04 $ IEEE I Eq. (7, O t, t ( O( t, O( t +, L, O( t ( = ad p, represets the k k -th recursvely sampled pot of the -th asc stroke put. he matchg proaltes of EP s ca e terpreted as the proaltes of gloal stroke postos ad those of IP s as the proaltes of local stroke shape dstortos. 5. Expermetal results 5. Data set I order to evaluate the proposed gesture recogto system, we collected data from 5 wrters. Amog them, eght wrters have some experece of usg the devce ad the others do ot have ay. he put devce was attached to a PC y usg the seral port terface durg data collecto. Sesor sgals were geerated from the put devce ad trasmtted ad saved the PC. Gesture laels ad represetatve shapes were show o PC scree. Wrters drew gestures whle lookg at the represetatve shapes. Each wrter wrote 3 classes of gestures y 24 tmes. hey were structed to hold the devce statc posto for a short tme ust efore ad ust after wrtg. Gesture shapes are teratvely desged for hgh recogto rates ad coveece of wrtg. At frst, we adopt gesture shapes from wdely used grafft o PDAs. Because of the lmted accuracy of traectory estmato algorthm ad the lack of vsual feedack to wrters durg wrtg, gestures wth smlar movemet hstory were cofused frequetly (the gesture par of 0 ad 6, ad 5 ad 8. hese cofusos are resolved y appedg a asc stroke to the ed of the gesture 5 ad 6. Fg. 7 shows the fal gesture shapes. gesture was draw durg the terval. he other tervals dcate that the devce was a statc posto. It s oserved that the accelerato s more sestve to the moto of the devce tha agular velocty from ts larger varato. accelerato agular vel YWG WG hÿg WG h G h G WG YWG [WG ]WG _W WWG YW [G YG WG ~ G ~ŸG ~ G WG YWG [WG ]WG _W WWG YW Fg. 8: Example of raw sesor sgals whe the gesture 2 s wrtte 5.2 Results of traectory estmato Fg. 9 shows traectores estmated from raw sesor sgals. exts o the left-top corer of traectores represet guesture laels. he shapes look somewhat dstorted from represetatve gesture shapes. Artfcal hooks are oserved start ad ed parts of traectores. he legth ratos etwee asc strokes are ot estmated relaly. It s caused y tegrato errors of ertal sgals ad also the lack of vsual feedack of traectores to wrters. otherless, traectory shapes look smooth ad atural, ad drectos of partal traectores are estmated relaly. Also, shapes of dfferet classes look dstgushale amog oe aother cacel delete eter Fg. 7: Gesture shapes for expermets. Fg. 8 shows a example of ertal sesor sgals otaed whe the gesture 2 s wrtte. he frst graph shows 3-axs accelerato sgals ad the other graph shows 3-axs agular velocty sgals. he large sgal chages from the tme 25 to the tme 75 suggests that the he shapes show ths paper are maly desged for testg the recogto performace of the devce. We have developed aother gesture set sutale for Vs. Fg. 9: Examples of traectores estmated from raw sesor sgals 5.3 Recogto results he recogto performace was measured y dvdg the data set of 5 wrters to four parttos accordg to wrters. Frst, the frst three susets were used for trag ad the other for testg. Secod, the ext three susets were used for trag ad the other for testg. I ths way, four dfferet cofguratos of trag ad test sets were used for evaluatg the wrterdepedet recogto rate. Fg. 0 ad show recogto rates y wrters ad y classes. he average recogto rate of all the wrters s 99.2%, ad all the wrters have recogto rates of more tha 96%. However, large varatos are show amog wrters. Amog gesture classes, the class 7 has the Proceedgs of the 9th It l Workshop o Froters Hadwrtg Recogto (IWFHR /04 $ IEEE lowest recogto rate ecause ts shape s smlar to that of the class. Whe t s cosdered that half of the wrters have o experece wrtg wth the proposed devce, the hgh recogtorate dcates the relalty of the proposed put devce. Recogto rate Fg. 0: Recogto rates y wrters Recogto Rate c d e Fg.. Recogto rates y classes ( c : cacel, d : delete, ad e: eter 6. Coclusos hs paper troduces the gesture recogto system 3-D space. he employmet of ertal sesors eales users to draw gestures almost ay place ecause they do ot requre ay exteral referece devces. I order to reduce the varatos of movemet hstores ad postures of the devce, the traectory estmato algorthm ased o ertal avgato system theory s employed to covert ertal sgals to traectores. Bayesa etwork ased gesture recogto algorthm s employed to recogze the estmated tractores. he proposed gesture recogto system showed a promsg performace; the average recogto rate of wrter depedet test was aout 99.2% o the dataase of 5 wrters ad 3 classes of gestures. It s qute a promsg result wth the fact that the half of the wrters have o experece wth the devce. he estmated traectores look somewhat dstorted from the orgal gesture shapes ut shapes of dfferet classes look dstgushale. he future work s to ehace the preprocessg step such as hook removal of the estmated traectores ad to desg gesture shapes more coveet to users. Ackowledgemet Mr. Kyugho Kag mplemeted the proposed hardware prototype y desgg PCB ad moutg sesors. Samsug Software Ceter desged the aesthetc remote cotroller case. Refereces [] L. Comerford, et. al, he IBM persoal speech assstat, Proc. of IEEE ICASSP, Vol., 200, pp 7- [2]
Related Documents
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!