#2 [state, video] Control the Tello drone! Node.js | TypeScript | JavaScript

Tello, TypeScript, Node.js

This is the second part of a series about Tello drone programming. This article provides information about the drone state & video stream. This time we won't discuss basic information, such as the connection to the drone, its host and ports, or UDP protocol— we did it in the first part of this series (link below)

Control the Tello drone! Node.js | TypeScript | JavaScript

In the next article, we will talk about how to combine stuff from previous parts and present them in the browser. So stay tuned! 👌 💪

All from the series

• #1 [overview, commands] Control the Tello drone! Node.js | TypeScript | JavaScript
• #2 [state, video] — this one.
• #3 [state, browser, websocket] Control the Tello drone! Node.js | TypeScript | JavaScript
• #4 — soon…

Table of content

#1 — State
#2— Implementation (state)
#3 — Example of use (console, state)
#4 — Video stream
#5 — Video display (POC)
#6— Conclusions
About the author
Repository link

#1 — State

According to the SDK, we can listen to the drone state on UDP port 8890 and get it in a string form. Thanks to this, we can easily get information such as speed, temperature, or battery level (full list below) in real-time.

Overview

The returned state is as properly formatted:

pitch:%d;roll:%d;yaw:%d;vgx:%d;vgy%d;vgz:%d;templ:%d;temph:%d;tof:%d;h:%d;bat:%d;baro: %.2f; time:%d;agx:%.2f;agy:%.2f;agz:%.2f;\r\n

• pitch: Attitude pitch — degree,
• roll: Attitude roll — degree,
• yaw: Attitude yaw — degree,
• vgx: Speed x,
• vgy: Speed y,
• vgz: Speed z,
• templ: Lowest temperature — Celcius degree,
• temph: Highest temperature — Celcius degree,
• tof: TOF distance — cm,
• h: Height — cm,
• bat: Current battery percentage — %,
• baro: Barometer measurement — cm,
• time: Motors on time,
• agx: Acceleration x,
• agy: Acceleration y,
• agz: Acceleration z.

Example state response:

pitch:4;roll:3;yaw:-47;vgx:0;vgy:0;vgz:0;templ:89;temph:91;tof:10;h:0;bat:88;baro:45.21;time:0;agx:78.00;agy:-60.00;agz:-996.00;

Getting state

As already mentioned the drone server provides listening for its state on UDP port 8890. To get the messages from UDP we will use the UdpSocket class which we created and discussed in the previous article.

IMPORTANT! To read the state of the drone, you have to set it into command mode first! I show how to do this in the previous article — Command mode section.

As a reminder, UdpSocket class looks like this:

So let’s move on to the practice! 💻👇

#2— Implementation (state)

A socket connection is done by UdpSocket class, so now we need to use it properly. Let’s create a DroneState class that will be responsible for listening and displaying the drone state.

In the class constructor, we create a new socket object and — through the addSocketEvents method — listen for two events: error and message. At this moment we do not need any other logic, so we simply display the received messages from the socket — actual drone state or error.

#3 — Example of use (console, state)

First of all, we want to observe the results in two consoles. In the first console, we’ll send commands to the drone — by using the DroneConsole class from the previous article. In the second console, we will receive drone state.

To do that, we need to run two different scripts on two different consoles. But before that, we need to create an object of our DroneConsole and DroneState classes. Let’s do it in each file.

• drone-contole.ts file

...
export class DroneConsole {
   ...
}
new DroneConsole(); 👈👈👈

• drone-state.ts file

...
export class DroneState {
   ...
}
new DroneState(); 👈👈👈

When we have all the pieces, putting them together is really easy:

1. Turn on the Tello drone.
2. Connect your laptop/pc/whatever-your-script-is to its WiFi. Tello WiFi is usually named like TELLO-xyz where xyz is a unique string.
3. From your first console run drone-console.ts file. (See Run script section below)
4. From your second console run drone-state.ts file. (See Run script section below)
5. From your first console send command: command.
6. At the second console observe the drone reactions! You can move the drone to see the state changing.

Run script

To automate the script execution a little I wrote simple scripts in package.json.

...
"scripts": {
   ...    
   "open:console": "tsc --outDir dist src/drone-console.ts 
                    && cd dist && node drone-console.js",
   "open:state": "tsc --outDir dist src/drone-state.ts 
                  && cd dist && node drone-state.js"
},
...

Each of them does two things: (1) transpile TypeScript code to JavaScript and places the resulting files in a dist folder, (2) go to the folder and runs transpiled script.

Now from your console, you can just run npm run open:console and from the other console run npm run open:state.

Received drone state

#4 — Video stream

Probably the most interesting part — video stream! According to the SDK, we can listen to the drone video stream at UDP port 11111. And that’s it — unfortunately… there is no more information about video there.

But! After some digging, I figure out that the returned stream is formatted in H.264/MPEG-4. (Un)fortunately, we won’t explain what it is and where it came from. What we need to know is that we need to decode this stream and we will be able to display it in the browser! We will do this in the next article — now, we’ll just check if the encoded stream is really a stream from the drone camera. And we will do it by FFMPEG solution — it would be our POC (Proof of Concept) that really works!

NOTE. If You would use our DroneState class just with the different port (11111) it would work too! But You would see some encoded message in the console — a bit of useless.

#5 — Video display (POC)

First of all, you need to download FFMPEG from their official website. I used version release 4.4 for Windows. It’s a proof of concept for drone video, so we won’t discuss how this tool exactly works. To test our video you need a few steps described below.

— — Step 1

The first step is to connect to Tello WiFi. Tello WiFi is usually named like TELLO-xyz where xyz is a unique string.

— — Step 2

Next, let’s tell the drone to start streaming video source. From your console run drone-console.ts script (see point 3 on Example of use section above) and send two commands:

1. command — setting the drone to a command mode.
2. streamon — telling the drone to start stream video.

Right now, the drone already streams encoded video packages on UDP port 11111.

— — Step 3

Display the video 😍 📹 Go to bin folder for FFMPEG you have already downloaded, where should be files: ffmpeg, ffplay.

From your console run the command below:

ffplay -i udp://0.0.0.0:11111

or

ffmpeg -i udp://0.0.0.0:11111 -f sdl “Tello COPTER”

Now — in a separate window — you should see the video comes from the drone!

If you see the video but with huge delays try to run the command below, where we additionally setting the probing size and framerate in bytes.

ffplay -i udp://0.0.0.0:11111 -probesize 32 -framerate 30

For more information check the official documentation.

Drone camera — real photos

#6— Conclusions

As you can see, receiving data like drone state and video stream with Node.js & TypeScript is quite simple! We learned the basic information about the Tello state and video: how to get it from the device, how to understand it, and how to display video in the decoding tool. In the next article of this series, we will create a simple app to display drone state on the website, so stay tuned! 😎

Thanks for reading and making it to the end! 💪💪

About the author

Passionate about programming and technology, especially related to Web Development. I’m currently working as a Web Software Engineer in Teleste Video Network.

Repository link

All of the presented code is available in my repository (link below). The code may be slightly different, but the idea stays the same.

pawelidziak/tello-copter