Artist, designer, programmer Cyril Diagne has tweeted a demo showing an AR app that can transfer items from the real world to Photoshop.
4/10 – Cut & paste your surroundings to Photoshop
Code: https://t.co/cVddH3u3ik
Book: @HOLOmagazine
Garment: SS17 by @thekarentopacio
Type: Sainte Colombe by @MinetYoann @ProductionType
Technical Insights: ↓#ML #AR #AI #AIUX #Adobe #Photoshop pic.twitter.com/LkTBe0t0rF— Cyril Diagne (@cyrildiagne) May 3, 2020
By taking advantage of machine learning and AI, Diagne was able to essentially copy items from the real world and paste them onto Photoshop on a Mac.
According to Diagne the secret sauce is BASNet, a tool designed for object detection and background removal. With BASNet, Diagne was able to replicate Phpotphop’s Magnetic Lasso Tool, “which detects edges of a physical object and frees it from its surroundings,” allowing users to transfer it directly to Photoshop.
Pasting the object to your screen was developed with the help of OpenCV SIFT, an image tracking technology able to detect where phones are pointed at the screen.
Diagne said that this was part of a series of experiments that explore how machine learning and AI make digital interactions more natural. He also expressed how he did not think about developing the tool further, however, after seeing the response to the demo, he’s considering developing the feature into a mobile app.
If you’re interested, check out Diagne’s GitHub page here.
Source: CultofMac
