Rumors Persist About Triple-Lens Rear Camera System With Advanced 3D Sensing on 2019 iPhones

While we're still two months away from Apple's widely rumored trio of 2018 iPhones, the rumor mill is already looking ahead to next year.

iPhone with triple-lens rear camera mockup via iDrop News

Much of the early discussion has centered upon the rear-facing camera on 2019 iPhones, with multiple reports claiming it will be a triple-lens array with advanced 3D sensing for augmented reality capabilities.

Taiwanese publication Economic Daily News says as much in a pair of reports today, claiming that the rear-facing camera on 2019 iPhones will feature "more advanced" sensing than the front-facing TrueDepth system on the current iPhone X, with a triple-lens setup capable of stereoscopic imaging.

With stereoscopic imaging, two of the sensors would be able to capture images of a single object from different angles. A triangulation method would then be used to obtain the distance between the iPhone and the object.

The third lens would also likely enable 3x optical zoom on an iPhone for the first time, enabling users to magnify the image in the viewfinder by up to three times without a blurry reduction in quality like digital zoom.

Bloomberg News last year reported that Apple aims to include rear-facing 3D sensing capabilities in its 2019 iPhones. At the time, Apple was said to be evaluating a time-of-flight approach that calculates the time it takes for a laser to bounce off surrounding objects to create a 3D image of the environment.

While the exact implementation remains to be seen, it's clear that the rear camera could play a big role in the augmented reality capabilities of future iPhones. Last year, Apple released ARKit, a framework that enables developers to incorporate augmented reality into their apps on iOS 11 and later.

Apple previews ARKit 2 during WWDC 2018 at around 22:05 mark of video

At WWDC 2018 last month, Apple previewed ARKit 2, with shared experiences like multiplayer games, persistent augmented reality, extended support for image detection and tracking, and more.


Discuss this article in our forums

Apple Announces New USDZ Augmented Reality File Format Coming in iOS 12

Apple today announced a new augmented reality file format called Universal Scene Description (USDZ) during its keynote event at WWDC in San Jose, California.


Apple explained that USDZ was developed alongside animation studio Pixar and is an open file format optimized for creating and sharing 3D Augmented Reality assets in a compact, single file.

Adobe VP Abhay Parasnis was introduced on stage during the keynote to confirm that it will have native support for Apple's new AR file format in its entire suite of Creative Cloud software applications, and will also be adding a new AR design app to CC with easy-to-use tools for creating new experiences.

More to follow...



Discuss this article in our forums

Gene Munster Shares WWDC Predictions: Beats Product With Siri Integration, Improvements to AI and AR

Ahead of next week's Worldwide Developers Conference, Loup Ventures analyst Gene Munster today shared his predictions for the features and services that Apple will unveil during the event.

Munster expects Apple to debut new Siri, AR, AI, and Digital Health functionality, including a Beats-branded accessory (presumably a speaker) that includes Siri integration, much like the HomePod. Some of Munster's predictions have been previously covered in rumors shared by Bloomberg's Mark Gurman, but Siri integration in a lower-cost Beats product is a new prediction.

We expect Monday's keynote to be highlighted by extending the reach of Siri (most likely adding new domains, opening HomePod to more capabilities, and integrating Spotlight), along with additional AI tools (new Core ML extensions).

We also anticipate new features around digital health (privacy and device management) and ARKit (development tools).

Expect Siri integration with Beats.

Collectively, these announcements advance the ease of use and intelligence of Apple's mobile and desktop experiences.
According to Munster, Apple may be planning introduce a $250 Beats-branded product that will offer Siri integration similar to the HomePod, allowing Apple to "advance its digital assistant ambitions" with a more affordable option. Apple currently sells a Beats Pill+ speaker for $179.95, and the device has not been updated in some time.

Apple is going to announce a new "Decade Collection" at WWDC according to a Best Buy leak, but that collection is limited to existing headphones in new colorways and does not appear include new products or a new speaker. It's possible that Apple does have a new Beats product ready to unveil, and a recent somewhat sketchy rumor did suggest that Apple's "low-priced" HomePod would be under the Beats by Dre branding.

That rumor would make some sense if Apple is indeed planning on introducing a Beats-branded speaker product that includes Siri integration. Siri competitor Alexa is available as an option on many speakers outside of Amazon's own, and Apple could be planning to follow in Amazon's footsteps. Obviously, though, it's not clear if the rumor and/or Munster's prediction are accurate.

Munster has several other predictions for features and services coming at WWDC. Specifically, he expects Apple to introduce new Siri domains, with support for "things like navigation and email" and integrated Spotlight Search to better improve Siri's performance compared to Alexa and Cortana, the AI assistants used by Amazon and Microsoft, respectively.

Munster also believes Apple will introduce new domains for CoreML, the machine learning SDK that Apple introduced with the launch of iOS 11. Munster doesn't offer details on what the new domains might be, but at the current time, CoreML offers features for developers like real time image recognition, search ranking, text prediction, handwriting recognition, face detection, music tagging, text summarization, and more.

Previous rumors have said Apple plans to introduce support for multiplayer augmented reality games, and Munster believes Apple will also introduce "subtle new developer tools" to improve AR development and lead to more compelling AR apps.

Similarly, rumors have indicated Apple is working on a Digital Health tool that will let parents better monitor the amount of time children are spending on iOS devices. Munster says Apple could also include additional features that notify users when data is being shared with developers and new device management features aimed at curbing "screen time and digital anxiety."

Munster's full range of predictions for WWDC can be read on Loup Ventures, and our iOS 12 roundup contains all of the other iOS-related rumors that we've heard thus far.

Related Roundup: HomePod
Buyer's Guide: HomePod (Buy Now)

Discuss this article in our forums

Google Maps Gaining AR Street View, ‘For You’ Recommendations, and More

Google today outlined several new features coming to Google Maps at its Google I/O developer conference in Mountain View, California this morning. First and foremost, this summer, Google Maps will be updated with a new personalized "For You" section that offers up recommendations based on your interests.

The Maps app is gaining a dedicated "For You" tab that is designed to "tell you what you need to know about the neighborhoods you care about." It will feature new restaurants opening nearby, trending spots, and more, based on places you've rated and places you've visited.


For You will be accompanied by a new "Match Score" that predicts which restaurants you're most likely to prefer based on your personal preferences, which is designed to help you to choose between multiple restaurant suggestions.

To facilitate making plans with others, Google Maps is gaining a feature that will let you long press on any restaurant or location to add it to a shortlist that can be shared with friends to get their input. Once shared, your friends can add additional suggestions to the list and vote on the place they'd most like to visit to find a group favorite.


Google also announced a new augmented reality feature that's coming to the Street View version of Maps for walking directions. When you're using Google Maps to find a location that you're walking to, you'll see an AR overlay with specific directions on which way to go and information on exactly what's around you, reimagining walking navigation.


Google says it is augmented GPS with VPS, or a visual positioning system meant to estimate a person's precise position and orientation using visual landmarks.

Google did not say when the augmented reality Google Maps features will launch, nor which platform they'll be available on, so it is unclear when and if we will see the augmented reality features in the Google Maps app for iOS devices. Given that most Google Maps features roll out to both Android and iOS, it's reasonable to assume that AR maps will come to iPhones and iPads.

Along with improvements to Google Maps, Google also announced changes coming to Google Lens, available in Google Photos and Google Assistant apps. Google Lens is gaining smart text selection, which will allow it to recognize text in the real world and then translate it to text on a device, so you can essentially copy and paste real world text.


Real-time functionality is coming to Google Lens, allowing it to proactively gather information about everything the camera is seeing to overlay live results directly on top of real-world items. A new Style Match feature will let Google Lens scan images and then find related items. So, for example, you can scan a lamp and find items in the same style, or scan an outfit and find similar clothes.


Google Lens is being integrated directly into the camera app on select Android smartphones like the Google Pixel and the new LG G7, which will give Android users quicker access to Google Lens functionality. All of these new features are rolling out over the course of the next few weeks.


Discuss this article in our forums

Snapchat Launches ‘Snappables’ Augmented Reality Games That Can Be Played With Friends

Snapchat today announced the launch of Snappables, which are new Snapchat Lenses that are designed to allow Snapchat users to play augmented reality games with their friends.


Interactive Snappable lenses are located right near other Snapchat Lenses in the Lens Carousel and can be accessed by tapping the icon on the left side of the capture button.

To use a Snappable, tap the Snappable icon and then follow the on-screen directions.

Some of the available Snappables let you create a dance party or a rock band with friends, while others are more game-like in nature and can be controlled through facial expressions, touch, and motion. You can play a game where you fight virtual aliens, for example, and then send a Snap to a friend to play along.


According to a Snapchat spokesperson, Snappables are the beginning of a new type of augmented reality experience on Snapchat. Prior to now, Lenses have focused on transforming your look or the world around you, while Snappables are sharable AR experiences.

In other Snapchat-related news, Snapchat parent company Snap is said to be planning to introduce new Spectacles later in the week. Little is known about the new glasses, and it's not yet clear if they will feature major design changes.

Snapchat can be downloaded from the App Store for free. [Direct Link]


Discuss this article in our forums

Developer ‘Survios’ Details Process of Porting VR Title to iMac Pro, Says Apple Has ‘A Lot of Passion’ for VR

When Apple launched the iMac Pro last December, virtual reality applications and content creation were headlining features for the high-end computer. Prior to that, the company focused on Metal 2 and VR-based content as major additions to macOS High Sierra during WWDC 2017, also announcing multiple game studios putting their support behind Mac-based VR experiences, like Valve and Epic using the HTC Vive headset.

Today, another VR studio backing Apple's efforts in this space has become the focus of a new report by Ars Technica. The studio, called "Survios," was approached by Apple to port its new software "Electronauts" to macOS, part of many VR apps aimed at showcasing the iMac Pro's top-of-the-line capabilities.


In the new interview, Survios studio head Mike McTyre and software engineer Jason Meisel discussed the "nearly painless" porting process, what working with Apple was like, and the important distinction between Apple's focus on AR vs VR.

According to Meisel, Apple focused on ease of portability and ensured that early VR Mac developers wouldn't hit roadblocks during the porting process. Right now on Mac, VR experiences are supported using an optimized version of Valve's SteamVR platform and Survios developed Electronauts in the Unity game engine, which "can build directly to a multitude of platforms" spanning macOS and Windows.
"Essentially, what they've done really well is that they've been working with Unity and with Valve to make that whole process of porting a game that already exists using Unity and just get it running on the iMac," Meisel added.
Speaking on the topic of the small install base of an iMac Pro-only VR title, McTyre said that the studio noticed "a lot of passion" from Apple, and it's clear that the Cupertino company is planning "a lot more growth" in VR moving forward.
We're seeing a lot of resources on their end, a lot of effort, a lot of passion. They want to focus on this and work on this. So that might be true now, but that's just the starting point. It starts here, and let's see what they add on to that next... I truly believe that they're going to put a lot more growth into that going forward beyond this. This is just a starting point.
To expand the support of VR on Mac computers, McTyre said that in a few years he hopes to see Macs launch with integrated GPUs that sport minimum spec requirements for VR. Right now, even the iMac Pro development kit includes an external GPU enclosure, and support for eGPUs will launch wide in macOS High Sierra 10.13.4. McTyre hopes this is just a stopgap solution for graphics-heavy apps like VR: "I do think we are not that far off from the built-in GPUs just being good enough to just, out of the box, play VR."

Photo of the Survios offices in Culver City, California by Samuel Axon via Ars Technica

McTyre also touched upon the difference between virtual reality and augmented reality, the latter of which has been more of a focus for Apple in recent years thanks to ARKit in iOS. Apple has been tied to potential future products that could use either AR or VR, but Survios is specifically focused on VR and McTyre said AR is just "not as far along yet as VR."
The concept that AR is a version of VR but better, in my mind, is not true. It's a different experience. Televisions did not make movies obsolete. Movies did not make books obsolete. Mobile gaming did not making console gaming obsolete, and console gaming did not make PC gaming obsolete. It's all nonsense. We've been through this a million times over the last century. It's a new medium, and AR and VR are different. Is there overlap, just like TV and movies have overlap? But they're not the same thing.
For more details on Apple's relationship with VR developers, be sure to visit Ars Technica and read the full interview with Survios.

Related Roundup: iMac Pro
Buyer's Guide: iMac Pro (Buy Now)

Discuss this article in our forums

Apple Joins Group Investment in OLED Microdisplay Company eMagin Amid Ongoing AR/VR Efforts

Apple has joined a group of companies investing in OLED microdisplay maker eMagin, believed to be part of Apple's overall virtual and augmented reality hardware strategy. LG, Valve, Immerex, and Stillwater Holdings have all joined in on the investment with Apple, which is said to be worth up to a total of $10.6 million (via TechCrunch).

eMagin confirmed the investment, stating that it will use the money for "working capital and general corporate purposes." The company created "a new kind of display" to use in VR headsets that allows for sharper images, which it believes will help fuel the "next generation" of AR and VR devices.

eMagin’s technology is notable in that it has created a new kind of display that can be used in VR headsets, which provides a sharper image by using a denser layout of lines (versus the pixels commonly used in existing products). This helps reduces the so-called “screen door” effect on the display and makes what you see up close through the headset much sharper.

“We believe that our direct patterning technology is a key differentiator for enabling next generation AR/VR hardware for the consumer and enterprise segments because of the brightness and the pixel density afforded by the technology,” the company notes.
For Apple, the last few years have seen a ramp up in the hiring of talent and acquisitions of companies that all revolve around its AR/VR efforts. Since 2015, Apple has purchased several companies in this area, but some acquisitions date back even further. A few of these AR/VR acquisitions made by Apple include companies like Metaio, Faceshift, Emotient, Flyby Media, RealFace, and VRvana.

Apple has already put the technology behind Faceshift to use with Animoji in iPhone X, but outside of software the company's plans for a hardware product in the AR/VR space remain unclear. Right now, plans for a pair of "smart glasses" or VR headset are believed to be in prototype stages, so if they do ever launch it will still be a few years down the road. More recently, a report suggested that Apple is ramping up development on an AR headset running a new operating system called "rOS."

Apple CEO Tim Cook has said that any AR/VR hardware from Apple is a few years away because "the technology itself doesn't exist to do that in a quality way." At the time, Cook said that one of the biggest challenges for building such an AR headset today would be the display technology required to make it work, suggesting the new eMagin investment could be helping Apple in this area. One of the last major reports regarding Apple's interest in AR hardware was the company's meeting with potential AR glasses suppliers, said to have happened at CES last month.


Discuss this article in our forums

Tim Cook Thinks Retailers Will Find Augmented Reality as Key as Having a Website

Apple CEO Tim Cook believes there isn't any sector or industry that will be untouched by augmented reality going forward.

Tim Cook via Alastair Nicol for Vogue

In a recent interview with Vogue, Cook said he believes the technology will transform everything from fashion runway shows to shopping.
"If you think about a runway show in the fashion world, that's a great application of AR because some of these, you want to see the dress all the way around, you do not want to just see the front." That kind of experience is all the more important now that runway shows are catering to a wider consumer audience watching online, and not just those seated in the front rows, he added.
Apple's chief envisions a world where customers will essentially be able to "point and buy" products. If your friend is wearing a pair of shoes you like, for example, you could point your iPhone at them, and a shopping app could instantly bring up information about the pair with the option to purchase them online.
"We don't have a plan to collect all of these objects, but I know companies who are working on that for their products," Cook said. "If you think about companies that offer a fair number of shoes, and [if a customer] sees a shoe and goes I want that one, you just want to point and [buy]. That will be a part of the shopping experience of the future, it absolutely will."
Cook is so confident in augmented reality's future that he believes the technology will become "as key as having a website" for brands.

Some retailers have already implemented augmented reality features using Apple's new ARKit platform on iOS 11. IKEA, for example, has released an app called IKEA Place that lets you virtually place furniture in your home, with true-to-scale models of everything from sofas and armchairs to footstools and coffee tables.


Cook thinks the current selection of augmented reality apps is only the very beginning of what's to come in the years ahead.

In one early ARKit demo, for instance, a woman is able to virtually try on various shades of lipstick and quickly choose her preferred color.


Later in the video, the woman browses a virtual aisle of images of herself with various cosmetics digitally applied, making her selection a much simpler process than the traditional hassle of physical makeup application.


Cook also said that the technology needed for augmented reality glasses "doesn't exist to do that in a quality way," suggesting that widely rumored Apple Glasses won't be released "any time soon."
"There are rumours and gossip about companies working on that, and we obviously don't talk about what we work on. But today I can tell you that the technology itself doesn't exist to do that in a quality way," Cook said. "We don't give a rats about being first, we want to be best in creating people's experiences. Something that you would see out in the market any time soon would not be something that any of us would be satisfied with."
Cook's sit-down discussion appears to be the same one that The Independent covered earlier this week.

Full Interview: Apple's Tim Cook On The Future Of Fashion & Shopping


Discuss this article in our forums

Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today

With the launch of iOS 11 today, Apple has turned hundreds of millions of iPhones into augmented reality-capable devices thanks to the support of a new developer framework called ARKit. With this technology, iOS developers can more easily craft AR experiences for users on compatible iPhones and iPads, using each device's built-in cameras, processors, and motion sensors.

As of now, the first wave of these apps are available for you to download and test on the iOS 11 App Store. The first apps range from game updates to practical everyday tools and even apps that encourage a healthier lifestyle, with more refined experiences likely coming in the future once developers get a grasp on what users enjoy with the first wave of apps.

Note that to use ARKit-enabled apps on iOS 11 you must have an iOS device with an A9, A10, or A11 processor. This means ARKit apps can be launched on iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, iPhone SE, iPhone 8, iPhone 8 Plus, and the upcoming iPhone X. For iPads, you can use the 9.7-inch iPad or the 10.5-inch and 12.9-inch iPad Pro. The older 9.7-inch model of the iPad Pro is compatible as well.

One you have iOS 11 installed on one of these devices, head over to the new App Store and check out some of the ARKit apps listed below to see how Apple's new augmented reality technology works in your own home.

Games


Splitter Critters (left) and Egg, Inc. (right)

Splitter Critters ($2.99)


- What's it about? Use swipes of your finger to split a colorful landscape and guide alien critters back to their spaceship, avoiding enemies and solving puzzles in the process.

- How's AR used? Scan a flat surface and then place a fully playable version of the main game into the real world, housed within a small white box.

Warhammer 40,000: Freeblade (Free)


- What's it about? Control an Imperial Knight war machine in the Warhammer 40,000 universe and fight the evil forces of Chaos through 170 single player missions using cannons, missiles, and thermal blasts to defeat your enemies.

- How's AR used? Drop your Imperial Knight from the main game into an AR "Photo Mode" to take snap shots of the war machine in the real world.

Egg, Inc. (Free)


- What's it about? A farming simulation game focused on hatching eggs, building hen houses, hiring drivers, and researching advanced technologies to upgrade your egg farm.
- How's AR used? Take a glimpse at your farm in AR with a "Farm To Table" picture-taking mode.

Thomas & Friends Minis (Free)


- What's it about? Build, decorate, paint, and create full train sets and then control characters from Thomas & Friends as you drive through your customized train set.

- How's AR used? Bring all of your creations into the real world with the app's AR mode, which places your train set on a flat surface so you can zoom in and around while still being able to interact with various tools and control characters.
Continue reading "Here’s a Look at the First Wave of Augmented Reality ARKit Apps Hitting the iOS App Store Today"

Snapchat’s World Lens Filters Now Include Augmented Reality Bitmoji

Snapchat will now allow users to view their Bitmoji characters in augmented reality, through an addition to its previous "World Lens" feature. With the update, Bitmoji will be able to do yoga, skateboard, drink coffee, and more, all taking place in real-world surroundings in AR (via TechCrunch).

Image via TechCrunch

Similar to Snapchat's popular dancing hotdog character, the Bitmoji will anchor to a spot in the environment and allow the user to walk around the characters for multiple angles and photo/video moments. They will be able to grow larger and smaller by swiping up and down on the screen, and multiple animation options will be available for users to choose from. Some of these will last longer than ten seconds, meaning Snapchat's multi-snap feature will be needed to string longer clips together.

Snapchat was featured briefly during Apple's September 12 media event, where Craig Federighi demoed a few face filters in the social media app. On iPhone X and iOS 11, Snapchat's filters -- and other AR experiences like the new World Lens Bitmoji -- will be improved thanks to the advanced front-facing camera sensors of the iPhone X, and ARKit in iOS 11.


The AR Bitmoji feature is rolling out globally on iOS beginning today, with Android users planned to receive the update sometime in the future. To be able to use the Bitmoji in World Lenses, users will first have to download the Bitmoji app and create their own character.


Discuss this article in our forums