DJI Phantom 4 – Computer Vision – Part 3 of our Review/Rating

In part one and part two of our first look at the DJI Phantom 4, we explored the improved construction and upgrading of the various assemblies. At first glance it would appear a Phantom 4 is a “perfected” Phantom 3 Pro. However, there is one major new innovation which makes this a revolutionary model- Computer Vision! This feature – and the current and future ramifications – will be explored in this post.

Computer Vision, Artificial Intelligence, Obstacle Avoidance – what does it all mean?

Before we delve into the specifics of the Phantom 4 system it will be helpful to discuss some background and definitions in the field of “machine vision”. The basics of this type of circuit have existed for a long time. For example, your oil or gas burning furnace uses a sensor to “see” whether there is a flame or not and will shut down the fuel if the flame fails to ignite.

Previous drones, including DJI models, have had three types of “sense and avoid” technology, all with limited usability.
Sonar – sound waves bouncing off a nearby surface – the first consumer drone, the Parrot AR Drone, had downward facing sonar. DJI improved upon it for the Phantom 3 Pro and Advanced and still includes it in the Phantom 4. When properly programmed, Sonar can be good at detecting large objects (like the ground) at a distance of up to 20 feet.
Infrared – some toy models used infrared sensors to detect obstacles. Due to various technical shortcomings, this technology is not well suited for more advanced uses.
Cameras – or Optical Flow – DJI (and Parrot) incorporated a downward facing low-resolution camera which was capable of picking up various patterns such as floor tiles, etc.
In combination with the sonar and other systems, this allowed drones to be flown indoors with greater precision.

The Phantom 4 breaks new ground by incorporating higher resolution cameras (4) on both the front and bottom of the drone body. These cameras feed information into a specialized processor to achieve the first true Computer Vision in a consumer drone. Those who want more technical background should visit the site of Movidius, the company which provides the processor for the Phantom 4. This systems is much more than a camera – it involves a system which actually learns (like our brains). This means that the Phantom 4 you buy today will likely improve in capabilities with future firmware upgrades.

This type of system is not a gimmick or toy – in fact, it’s at the forefront of a revolution which is on the cusp of changing our world. Self driving cars, for example, will use similar technology and save millions worldwide from death and injury on the roads. Uses for computer vision are almost endless – especially since these systems can learn…meaning they will eventually be smarter than most of us at accident avoidance and object recognition.

Getting back to the Phantom 4, here are some ways the new systems improve this DJI model.
1. The bottom facing cameras have been improved – which, along with the sonar, help even more with indoor flight and ground sensing in any environment. They also help the drone maintain a very stable hover – much more stable than using the other sensors (barometer, GPS, IMU) only.
The Phantom can also use these in combination with the front sensors to determine how to properly clear object such as fences, vehicles, boulders, etc.
2. The forward cameras are capable of many tricks – among them:
a. Avoiding obstacles in the way of the Phantom.
b. Recognizing and tracking (following) objects such as people, bikes, cars, boats, etc.
c. Allowing for “point and fly” on screen control which also includes avoidance and/or recognition of objects.

DJI has elected to name these features as follows:
Obstacle Avoidance or Sensing (OA) – allows the Phantom to stop or fly around objects.
Active Track – allows the Phantom 4 to “see” a moving target and follow it in various ways.
TapFly – Point at a faraway point on your screen and the Phantom will fly there – avoiding obstacles along the way. Example – point at a Church Steeple in the distance and the Phantom will go there, while flying around large trees which may be in the flight path. Another use might be to fly under a large bridge toward an object in the distance – since you don’t need to actively control the Phantom during this process, you can concentrate on the camera to obtain the best footage or pics.

DJI has implemented these systems in a logical fashion so as not to confuse the pilot. OA can be turned on or off, but most pilots will elect to leave it on. The bottom cameras and sonar are always on. The Active Track and Tapfly modes are “Intelligent Flight Modes” and require switching modes as well as input from the pilot before execution.

As with airbags and seatbelts, the most useful safety innovations are often those which work in the background – and this is the function of the main OA system. The settings page from the DJI Go App is shown below:

At first glance these settings can be confusing – which is why we underlined (red) the functions that each apply to. The topmost setting, Forward Obstacle Sensing, is usually turned on and allows the Phantom 4 to operate in the manner shown in the notes below it. The other settings only relate to when the operator is using ActiveTrack or TrackFly (intelligent flight modes).

Forward Obstacle Sensing works as shown in the pic below – the Phantom 4 senses obstacles in front of (and below and above at certain angles) and will note the approx. distance to them. It will slow and then stop before hitting the obstacle. The distance at which it does so depends on the speed of the Phantom – it will slow sooner if flying fast, and later (closer) if flying very slow.

The Phantom 4 has a wide field of vision – but pilots should keep in mind this does not include all possible angles. The graphic below shows the angles involved.

Note: Newer and inexperienced pilots should not attempt to use these advanced modes until they become very familiar with the Phantom 4. Carefully read the owner’s manual and watch online videos explaining these functions.

ActiveTrack – Follow me using Computer Vision

While other drones have “follow me” modes, the DJI Phantom 4 uses a different method for doing so. Previous models used GPS – one in the Drone and one in the controller (or on a watch band, smart phone, etc). The combination of two devices that know their position(s) allows to them to maintain a distance from each other as well as know the direction in which to move and point the camera.  This system will work but is subject to the rather poor accuracy of GPS – each of the devices could be off as much as 10 feet horizontally.

The Phantom 4 can track objects without the use of a second GPS device – in other words, the subject being track does not need to carry a device or controller. Instead, the Phantom 4 front facing cameras see the person, vehicle or other object and recognize it as it moves.  A shot is set up in the following manner:

1. Start Phantom and raise to at least 10-12 feet off the ground. Switch into the intelligent flight mode screen and select ActiveTrack.
2. Adjust the Phantom camera to place the object in the approx. center of the Go App screen and use your finger to draw a rectangle around the object. The Go App will recognize the object and a round GO button will show on the screen adjacent to the object being tracked.
3. Hit the GO Button to start ActiveTrack – hit the red STOP button or the “Intelligent Flight Pause” hardware button on the top left of the Remote to exit the tracking.

The video below shows the ActiveTrack, TapFly and the Sport (speed) modes of the DJI Phantom 4 Camera Drone.

TapFly is a mode I probably won’t be using much – it is more suited to those who have trouble with the “sticks” and other flight modes. Pointing and flying is simpler than actually flying – but, as we have seen with the recent Tesla crashes, the technology for fully autonomous and safe flight may not yet be fully mature.

ActiveTrack will be useful in many scenarios – but, once again, the operator needs to keep in mind that obstacle avoidance is not yet perfected. Ideally ActiveTrack will be used with an operator always at the Remote.

Another new mode of the Phantom 4 is “Sport Mode” which allows for a higher speed flight than former models (up to 45 MPH).

Summary and Recommendations

The DJI Phantom 4 is the best consumer level (under $1500) Camera Drone available on the market. This rating is based on features, camera quality and overall reliability. In addition, the price is reasonable – about $1200 currently. Those on a tighter budget will be more than satisfied with the lower priced Phantom 3 models – the Phantom 3 Advanced is less than $700 and contains most of the same features other than the obstacle avoidance (computer vision).

The best current prices on the DJI Phantom 4 and DJI Phantom 3 models can usually be found at Dronesavings.com – a Drone price tracker.

Samples

Here is a video of some Phantom 4 footage:

Still Picture Gallery – Click 2x to enlarge. (pics are downsized even when viewed full).

Thanks for Reading! If you have interest in the purchase of a DJI Phantom please see the links below:

DJI Phantom 4 – At the worldwide DJI Store – Free Shipping