Blink an LED over the internet: the begginer’s low level approach

ledoth

In this article we continue our road to the IoT approached at a low level, or better, at bottom-up approach, that will be applied in the future to my UART interpreter. Here I will talk about how to drive an LED over the Internet. So as always I’m using Python libraries to get things done safely, quickly, but also make them useful under the knowledge point of view. So, if you are trying to do similar things, this can be an article that is not completely useless. And remember, if you can blink an LED, you can do whatever you want… This is the “Hello World!” of the basic IoT. Maybe even less.

But be aware, there are a lot of guides that allow you to do the job quickly. You are reading this because I am developing an hardware platform which works differently from most of ready-to-go solutions, and I usually need to develop my own application and if something become useful, I write about that, like in this article.

Let’s start

As a starting point, you have to understand what you need. Here, since I don’t have any ethernet/wifi adapter to be used by an MCU and considering that I may need some calculus power, I used the well trusted RaspberryPi B. I set it up with a basic Raspbian distro and configured it to use a static IP address local to my LAN. Forwarding the traffic from outside is made by the router connected to the internet.

If you start from scratch and don’t have the DHCP disabled and the RPi doesn’t have a fixed IP, you can connect it to the LAN and observe by looking at the router at what address is connected to. Then to access to the board once you know the IP, you should have the ssh daemon running, this is the default normally. In Windows, I use PuTTY to access to it.

Once entered in our RaspberryPi at its new address, we set a static IPv4 address (note that in the so called “IoT” stuff the IPv6 should be used, but for now it is not a big problem) to avoid checking the IP every times. Open the file /etc/network/interfaces, it should look like this:

auto lo
iface lo inet loopback
#uncomment to change loop address
#address <address>
#netmask <address>

auto eth0
iface eth0 inet static
address 192.168.1.5
netmask 255.255.255.0
#uncomment for use RPi as router
#network <address>
#broadcast <address>
gateway 192.168.1.1

where iface eth0 inet static matters, because in this way we assign a static local IP address w/out using the DHCP of the router. Remember to set the IP accordingly to the gateway, where in this case represents the address of my router. The netmask should be identical.

Restart the network typing on the console:

sudo /etc/init.d/networking restart

and may not work, in that case simply reboot the RPi. Now, every time you boot the Raspberry and connect it to the router, it should always have the address that you have choosen, in this example is the 192.168.1.5

The scripts

As usually, I wrote a script that should run on the RaspberryPi, called server and another client that you can run on whatever PC, smartphone which can interpret python commands. Those are derived from the same functions described in this article, and with a new one, fine_data(), to refine the data received by cutting out the number of byte attached at the head of the string prior to send the data. Be aware that the server implemented is not working like a real server, it is capable to handle only one connection at a time, because my application because I want it. It’s a kind of  be-or-not-to-be-server. Otherwise, the basic concept is that you need to put the server.accept() call in a loop and exploit multithreading. That is why servers are needing high parallelism and memory to allow parallel execution, instead of single thread performances.

Both scripts uses an infinite loop to interpret the data, and the client send to server whatever command is inserted from the user. The server itself should verifiy the command and decide what to do. So what’s new, now, is in the server, where thanks to the library GPIO, prepare and drive one pin of the RaspberryPi. Depending on what is received from client, the pin 3 of the RaspberryPi (in my case is model B, rev 2) can be set to high or low. The server reasponds to client according to the action taken with the send_sck(clientsocket, msg).

Something like this:

while 1:
cmd = 0
mex = receive_sck(clientsocket) # lettura comandi veri
try:
mex = fine_data(mex)
print(&amp;amp;amp;quot;Incoming data from client: &amp;amp;amp;quot; + mex)
if mex=='on':
msg = 'Led ON.'
msg = msg_len(msg)
send_sck(clientsocket, msg)
cmd = 1
GPIO.output(3, 1)
if mex=='off':
msg = 'Led OFF.'
msg = msg_len(msg)
send_sck(clientsocket, msg)
cmd = 1
GPIO.output(3, 0)

if mex=='q':
#print(&amp;amp;amp;quot;Incoming data from client: &amp;amp;amp;quot; + mex)
msg = &amp;amp;amp;quot;Server ended. Stay rock.&amp;amp;amp;quot;
cmd = 1
msg = msg_len(msg)
send_sck(clientsocket, msg)
GPIO.output(3, 0)
break;
except:
print(&amp;amp;amp;quot;Uncorrect format received&amp;amp;amp;quot;)

if cmd == 0: # is still zero
msg = 'Unrecognized command.'
msg = msg_len(msg)
send_sck(clientsocket, msg)

server.close()
print(&amp;amp;amp;quot;Server connection closed.&amp;amp;amp;quot;)

Note that this GPIO library “simply” interacts with hardware control registers of the BCM2835 by accessing the file /dev/mem, and since it is a system file you must run the server script as a root. In the server script you should locate the line below and set the address and port like this

server.bind(('192.168.1.5', 8000))

while in the client

client.connect(('192.168.1.5', 8000))

The server should have the internal LAN address. The internal port, 8000 in this case, is the one used inside the LAN by the server. At the moment in which you decide to access from outside, you should open a port (can be different if you want) on the router allowing it to redirect the packets toward the internal address:port, here 192.168.1.5:8000. For this capability, refer to your specific router manual. About the port, an high number (> 1024) usually do not provides standard services, so often conflicts are not encoutered.

The client.connect() must use the address of your ISP provider if you are outside the LAN, and the proper port opened on your router previously. Here are the same because I’m using the scripts inside my LAN, so no redirect is needed. Note that you can’t test the correctness of forwarding from inside the same LAN where the server resides. If you are in the same LAN, you can instead connect to localhost and run both scripts on the same machine.
There are tons of guides on emulate a free static public IP and I don’t want to load too much this presentation. And since I’ve not tried this method in first person, I will skip the static IP tricks for now.

You can download the scripts and start playing with them. I’ve remained quite short in descriptions because I want to arrive at the next part. 🙂
If you have questions, you can ask them in the comments, of course!

And finally…

The final result is a LED which is on whenever I write “on” and viceversa if “off”. Connect using an SSH connection to the board and start the script “server.py” by typing

sudo python3 server.py

because I wrote it in Python3. If there is an error about an already used port:

[Errno : 98] Message Address already in use

it’s normal, maybe there is an open connection or if you are already ran the script, maybe the client has not closed the connection. In that case you can change the port: after few minutes that previous port will became again available.

I remotely driven the GPIO 3 (SCL) pin. Be careful to follow the guidelines  to avoid to ruin the SoC pins, and remember that those pins can output up to 50mA. See image below:

The setup is very trivial, just the GPIO 3 with a series resistor connected to the LED. This useless picture is posted “just for fun”. 🙂

ledThe server script has a console allowing to understand what is going on, while the client works as always in a console style. Whenever the LED is in the wanted state, the server confirm that by printing on screen. The use of the screen can be avoided if not needed, because the response message is sent to the client, as you can see.

serverThe client, for now, requires the user intervention, so the screen is needed to allow writing commands in console, as below.

client

 Downloads

Client:  https://www.dropbox.com/s/xzxoirvgg2hwm2u/client.py?dl=0

Server: https://www.dropbox.com/s/o6b0klsoqcymch8/server.py?dl=0

Network programming – an introduction

As a support towards a complete tutorial and presentation of the “An UART interpreter“, I discovered the need to learn something about networking under Python language. In this way, I should be capable to enable the access to Internet of my microcontroller based board.

So I will start by showing an example on how to send any signal to the other end of a network – here in the same localhost, i.e. send messages from my PC to itself, but once the mechanism is set, we can speak to the other side of the planet by simply changing the IP address. By doing this, the UART interpreter will be IoT-enabled.

For everyone who don’t know anything about networking softwares and modules and wants to learn something, like me, the best thing is to start from the bottom layer of the network: sockets. Which are not so bottom, but let accept this trade off. In these examples is developed a sample software in Python 3 running on two machines, here used the same PC speaking in local, i.e. the address 127.0.0.1, or localhost. I used Python because is well suited to easily work with the RaspberryPi, which will be exploited soon in next projects. And can be used to test the set of networking scripts described here.

What is a socket? Basically it’s an “istance” of two point of communication between processes, or in other words are an IPC methodology (Inter-Process Communication). You can use them on local machine or between machines separated by any kind of networks. Of course, there are more efficients methods to allow two processes to communicate on the same machine, but sockets are almost the only simple solution to allow communication over the network. From the user point of view, the socket represent an IP address to which you want to connect to by using a specific “port” that can (but don’t have to) be different depending whether it is a request or an actual data response. A port then can be considered as an instance of communication with that IP, which can be treated in a different way as different as the port chosen, for that reason you should not use ports with a fixed function and causing possible conflicts between processes. Use for example:

127.0.0.1:8000

Thanks to sockets, you can also see this website, where data pass through your IP at port 80. For a complete description I invite you to search around, since explaining them deeply is not the purpose of this article and I’m not capable to.

Server side

In Python, we need to import the module socket in order to use a server socket and binding it at localhost address. This is the server address, but can be anything else than localhost (127.0.0.1). Then we listen for a certain number of connections at a time, here just one connection at a time is allowed, but typically is 5. The socket is an INET type  since we deal with internet protocol addresses and is STREAM because I want to keep monitoring of the activity until I decide to disconnect from it. If this is DGRAM, I send my data and then disconnect. But here I want to disconnect at the end of the client execution only.

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 8000))
server.listen(1)

Then in a loop we need to accept the connection, creating a client socket with its address, then receive from the client a string saved in mex and send the server aknowledge to the client.

while 1:
 #accept connections from outside
 print(&amp;quot;Server notification: waiting for client&amp;quot;)
 (clientsocket, address) = server.accept()
 mex = receive_sck(clientsocket)
 print(&amp;quot;Server notification: \n\tIncoming data: &amp;quot; + mex)

 msg = &amp;quot;Server notification: data received&amp;quot;
 msg = msg_len(msg)
 send_sck(clientsocket, msg)

A sad but true thing is that inside blocking sockets I can wait forever because there is not a prefixed size. So I’ve implemented by using the msg_len() method a protocol which attach before the message the number of chars followed by an underscore character. Of course, the number include the number of char used by the number it self, too. The method return the modified string.

In this way at receiving part I used the receive_sck()  to receive and check if in the received part the “_” is present, thanks to this I can identify the number. After that I set the MSGLEN to this number, exiting from the while after that amount of data is received, and if present, more data is discarded since is not part of the message, according to this protocol. Must be noticed that even if the maximum buffer allowed is 2048 chars, I can receive even only one char at a time, if the line is congested. So a marking char must be searched every time I receive from socket until I find it. The marking char here is “_”.

With the send_sck() is more simple, because what I send is known, but must be previously transformed to attach the number of chars, useful at the receiving part. When transmitting an error handling can be implemented.

Client side

On the other side of the connection the same methods are reused for modularity. They can be improved by using heavily a self created class, which I didn’t done here.

client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('localhost', 8000))
msg = 'Hello IoT!'
msg = msg_len(msg)
send_sck(client, msg)
client.shutdown(1)
mex = receive_sck(client)
print(&amp;quot;Client notification: \n\tIncoming data: &amp;quot; + mex)
client.close()

Here I’ve implemented the socket client and connected it to server, the message msg is sent to the client socket called with fantasy client , sending the content to the server at localhost, i.e. the server address. The new thing is the client.shutdown(1), in which I close the client transmission capability allowing it to still receive, just to check the aknowledge from the server. Then I close the socket, disconnecting from server.

Conclusion

As a result, this is how two separate processes (server.py in the white backgroung and client.py executed on prompt of Windows) are communicatin each other. Client script is executed after server is waiting for client. Raw data received from each part is printed on stdout after the “Incoming data: ” and the actual payload contains the number of chars added by the protocol. These can be filtered out when printed, because are needed only by the algorithm and usually not by the user.

serverclient

Here you can dowload both client and server and play with them.

Source Code:
https://www.dropbox.com/s/766bokip1wogr0o/client.py?dl=0
https://www.dropbox.com/s/8fz2yz352r58ut1/server.py?dl=0