Instant Expert: Secrets & Features of iOS 10 | iLounge Article


Instant Expert: Secrets & Features of iOS 10

Click Below to Read the Rest of This Article:  


Although Photos skipped a major enhancement with last year’s release of iOS 9, it appears that Apple is now making up for this lost time in the iOS 10 Photos app with new intelligent photo analysis features akin to what we’ve seen in Google Photos — but with the huge added privacy benefit of performing all of the magic directly on the user’s iPhone, iPad, or iPod touch, rather than in a cloud-based data center.

Face, Object, and Scene Recognition

The Photos app gets a new piece of magic that the company has dubbed Advanced Computer Vision — ultimately a fancy name for deep learning that is going on under the hood to provide the user with intelligent recognition of faces, objects, and scenes right on the device.

Facial recognition isn’t entirely new — the macOS Photos app (and iPhoto and Aperture before it) has been able to do facial recognition for some time. However, for whatever reasons Apple never chose to sync this information to its iOS counterpart — even after iCloud Photo Library came along to keep everything else in sync. In iOS 10, however, instead of merely syncing faces recognized in the macOS Photos app, the Photos app can now perform facial recognition directly on the user’s iOS device.

However, the new algorithms go well beyond just recognizing faces — Apple has built in recognition for objects and scenes as well, and the really unique part of it all is that the iOS 10 Photos app is doing this all directly on the device, which will be a huge bonus for anybody reluctant to turn over their entire photo collection to analysis by Google’s cloud-based algorithms.

As a result, the organization of photos and the search experience has been vastly improved. Faces are now grouped into People albums, and you can now search for faces, places, objects, and scenes to narrow down your results. The object recognition is surprisingly good, and common objects will show up as pre-defined “categories” when searching — for example type in “bunny” and you’ll get an option for “Bunny Rabbits” and “cars” will provide options for not only standard cars but other more specific categories like “sports cars” and “trolley cars.” Of course, what you’ll see in these suggestions will depend largely on what types of photos you have in your library.


Apple has taken Advanced Computer Vision one big step beyond just providing advanced searching capabilities, however, adding a new feature known as Memories, using deep learning to algorithmically group photos and videos into relevant collections based on people, places, things, and times when photos were taken.

For instance, Memories can include groupings such as “On This Day” to show a retrospective of pictures taken on the same day in previous years, “Best of” collections for photos taken over a series of several months or a year, location-based groupings for vacations, event-based groupings for holidays, and more.

A new Memories tab at the bottom of the Photos app provides access to recent memories, beginning with memories selected for the current day at the top and working back chronologically as you scroll down.

Selecting a specific memory takes you to a detailed view that includes an automatically generated movie for that memory and a summary view of all of the key photos from it. A Show All button lets you switch to a view of all photos for the memory or back to just the Summary view, and you can scroll to the bottom of the memory to view a People section showing the faces that are found in the photos in that memory, a Places map showing where the photos were taken, and a Related section providing quick access to other related memories — generally those that involve the same people, were taken in the same places, or at the same time of the year. You can also use options at the bottom to mark a memory as a Favorite or remove it from your library entirely.

A new Show Holiday Events switch can also now be found in the Photos section of the Settings app, allowing you to decide whether Memories are automatically created and labeled based on holidays in your home country.

Memory Movies

Tapping on the video thumbnail at the top begins playback of an automatically generated movie for the selected memory, which will include a slideshow of key photos and video clips set to music.

The automatically generated videos work pretty well, although iOS 10’s algorithms can only guess at things like length and mood music. Once the video stops playing, sliders appear that you can use to adjust both the mood of the video along with whether you want to generate a short, medium, or long version.

An edit button at the bottom — which is now represented throughout the iOS 10 Photos app as three sliders — allows you to perform more advanced customization, including changing the title wording and font or choosing the soundtrack. You can select music from your own library, set the specific length of the video clip, and choose which specific photos get included or excluded. Think of it as “iMovie Lite” with most of the initial composition work already done for you in advance.

Expanded Details + Related Photos

A Details section now appears below individual photos, similar to the one shown for Memories, which will show you the people identified in the photo, the location where the photo was taken, and Memories related to the photo. You can tap on any of these to go directly to a person view, find other photos taken nearby, or go directly to a related memory.

Similarly, tapping on a heading in the main Moments timeline now displays a similar Memory-style detail view, rather than just taking you straight to a map view as it did in iOS 9.

New Smart Albums

A few new special albums have also been added in iOS 10. While Memories gets its own tab in the bottom navigation bar, features for Places and People become new smart albums that appear right after Favorites in your Albums list. And if you’ve marked at least one memory as a favorite, you’ll also get a Favorite Memories album.


In prior versions, the Photos app allowed you to select a map view for individual photo groupings, but iOS 10 now allows you to view your entire photo library on a map via the Places album.

The map view begins with the widest view necessary to show all of your photos, and as you zoom in, the larger groupings of photos will expand to show smaller groupings of photos in more detailed locations on the map. You can also switch to a Grid view if you’d prefer to just see Moments-style groupings of photos for the selected area of the map.


People provides a collected view of all of the faces that Photos has detected within your library. Individual faces can be added as favorites to highlight them at the top of the album, and a Show Favorites Only button allows you to hide other faces. Tapping on an individual face provides a view similar to the one used for Memories, with an automatically-generated summary video at the top, a summary of key photos below, people, places, and related memories. Options found near the bottom provide the ability to favorite or unfavorite the person, confirm additional photos for facial recognition, and create a memory out of the person’s view.

The Confirm Additional Faces option takes you to a screen-by-screen preview of other possible faces, with the proposed face highlighted and Yes/No options to let you confirm whether or not the face matches the current person. If you’re viewing a person who has not yet been identified with a name, you can tap at the very top of the Person screen to type in a name; merging two different people entries is done by entering the same name for both — Photos will ask you to confirm that you want to merge them, after which they’ll be combined into a single Person entry. iOS 10 appears to add suggested faces based on how frequently they appear in your library, so the good news is that you won’t get entries automatically for every random person who appears in the background of your photos. You can add these manually by tapping the Add People button and browsing through the list of all detected faces, and if you’ve got some on the list you don’t want to see, you can hide a face by tapping the Select button in the top-right corner, selecting the face(s) you want to hide, and then tapping the Hide button at the bottom. Options also appear at the bottom to mark selected face(s) as favorites or merge two or more selected faces.

Editing Improvements

The auto enhance feature has also been improved in iOS 10, and some new editing features have been added. The Edit button is now displayed as an icon of three sliders, and has been moved from the top-right corner down to the bottom bar to make room for the Details button at the top.

The light editing controls now include a Brilliance slider, and the built-in Markup feature — previously only accessible when working in the Mail app — can be accessed from the plug-ins section.

Home + HomeKit

When Apple debuted HomeKit two years ago, it was merely as an underlying framework that third-party home automation accessories and apps could tie into. HomeKit had virtually no user-facing interface beyond Siri and whatever third-party apps could provide. As the platform matured over the past year, we quickly came to the conclusion that this was a missing piece to the HomeKit experience. While there were some great third-party apps out there, many of them spoke to a developer or manufacturer’s own vision for the platform, and few engendered the simple and accessible user interface design that was typical of Apple itself.

Now that the HomeKit platform has come into its own, Apple has decided to step it up in iOS 10 and provide its own core app for the service, with tie-ins to Control Center for quick access. While the new Home app isn’t comprehensive by any stretch of the imagination, in our opinion it’s a good everyday “home base” for most users for simple and intuitive control of HomeKit accessories.

Using the new Home app, you can add, manage, and control HomeKit accessories individually or via scenes. The basic functionality of Home isn’t all that different from many of the good third-party HomeKit apps we’ve seen, but Apple presents a nice layout, with a summary view of temperature and sensor data at the top, as well as lights and devices that are on. Any device or scene can be marked as a “favorite” to include it on the main screen and in the Control Center card, and you can choose to include or exclude specific devices from the “status” screen at the top.

A Rooms section provides access to groupings of accessories by individual room, which can be navigated by swiping left and right or bringing up a list of rooms from the menu button in the top left corner. Each room — and the main “home” screen — can also have a customized background image set to help identify it. Each room will contain all of the accessories located in that room, as well as any scenes that include accessories in that room.

The Automation section is used to set up rules to activate HomeKit accessories based on location, time of day, actions from other accessories, or conditions detected by sensors. The rules that can be configured here aren’t nearly as sophisticated as some of the better third-party HomeKit apps we’ve seen, but again it’s a starting point, and Apple has done a good job of making the rule creation process simple and intuitive.

The Home app also provides interactive, rich notifications, which will provide options to control accessories — such as viewing a doorbell camera and unlocking a door when the doorbell rings. Similarly, interactive notifications can be used for automation triggers that might otherwise require authentication, such as a rule to unlock a door when approaching your house.


Click Below to Read the Rest of This Article:  


« Instant Expert: Secrets & Features of tvOS 10

Inside the betas: iOS 10 Photos gets Advanced Computer Vision »

Related Stories

iLounge Weekly

Recent News

Recent Reviews

Recent Articles

Sign up for the iLounge Weekly Newsletter

iLounge is an independent resource for all things iPod, iPhone, iPad, and beyond.
iPod, iPhone, iPad, iTunes, Apple TV, Mac, and the Apple logo are trademarks of Apple Inc.
iLounge is © 2001 - 2018 iLounge, Inc. All Rights Reserved. Terms of Use | Privacy Policy