HardWire, the enhanced Arduino Wire library

cut_ST-3402W-HD.jpg

I have spent a lot of time on playing and later working on the I2C buses. I almost always used self developed libraries, both for learning and job requirement purposes. Then, when experimenting with a portability of a project in the Arduino environment, I started to trying the official Wire library, discovering some incompatibilities for my needs.

When the limitations were somehow fixed, even though the small number of modifications (it is somehow a sort of hacking of the library) I found that the my final result were so useful to me that I thought was worth to write a short article about it. But let’s start with the problem found in the Wire.

Arduino real-time buffer problem

I realized that if I wanted to make an Arduino slave, when receiving from master, it actually reads from a buffer that is already previously received. This make the communication not byte to byte controlled as I want it to be for my need.

In other words, basically when you read:


while (1 < Wire.available()) { // loop through all but the last
char c = Wire.read(); // receive byte as a character
Serial.print(c); // print the character
}

means that you will read everything that was already sent from the master. So you cannot signal to the master to stop the ongoing physical communcation in case the slave cannot receive more bytes, or you cannot take decisions depending on what byte is received.

But more generally, in this way the Atmega328 (what I have used) cannot behave like any dedicated I2C slave device: i.e. like an RTC clock, an EEPROM, which actually are making their decisions based on the current byte currently received from the bus, not after such transaction has been finished. I wanted to do the same.

In other words…

A slave Arduino cannot behave like any other dedicated slave device: this is not fair. Join the cause.

The solution is to have a library that allow me to do things in the hard way. The wire made hard. The wire more close to the hardware in a simple way. I called it the Hard Wire. Because somehow allow me to made the microcontroller to behave as I want at very low level in the I2C communication, obtaining the equivalent of an hard wired slave device.

I’ve just added to the Wire library various “probes” inside each step of the I2C data exchange. As you may know the I2C data exchange is made my many steps, where the START and STOP are just the starting and ending point of a bus cycle:

command.gif
A typical I2C bus cycle in which are transmitted (or received) two bytes

The hardware of a slave can interact and decide to do something after a START condition, ADDRESS, R/#W mode, choose if send or not the ACK, decide on run time what DATA to send and so on. When doing complex stuff and being interfaced as a slave in the I2C, can be useful to decide an action during these I2C bus steps.

With the conventional Wire library, you can take action only after the STOP or when addressed. In my projects, when used an Arduino, I need to take different action potentially every time I am in the following states:

  • Addressed in slave read mode
  • Addressed in slave write mode
  • ACK received after every byte sent
  • NACK received after byte is sent
  • ACK sent
  • START
  • STOP
  • RESTART

Then I decided to keep things simple, to re-adapt the already tested Wire library, which already handle the underlined green states.

Arduino real-time solution: runtime control

To countermeasure those kind of problems, I have placed functions inside each state of the I2C bus (those functions are the handlers, called when such I2C state happens). So that, over than the usual functions, the user can set up a fully working slave system implementing a few of other function than the usual onReceive and onRequest, where a particular action can be taken just after a byte is received or sent from the slave, and not only after we have reached the STOP condition. All by still using the same methods of the official Wire library!

How to use it

Handlers

Basically you can follow the official Wire guide. But this hard version also supports the following handlers assigned during the initialization (are listed also the official ones):

  •  Wire.onReceive(handler) -> handler takes the number of bytes received, here the data buffer can be read, since the master has terminated the communication (official from Wire)
  •  Wire.onRequest(handler) -> handler takes no parameters, but when called the slave prepares data to be sent using the standard Wire methods available, like read() and write() and whatever (official from Wire)
  •  Wire.onReceiveData(handler) -> handler takes last received byte for the slave runtime computation, but will also fill the data buffer with that byte, as the official Wire does transparently (not present in Wire, new in the WireHard)
  • Wire.onReceiveDataNack(handler) -> handler takes no parameters but is used to notify the slave, since this received byte shall not be used (not present in Wire, new in the WireHard)
  • Wire.onReceiveAdx(handler) -> handler takes no parameters, but the slave is notified to prepare itself to receive data in the next bus transaction, the official Wire does it transparently (not present in Wire, new in the WireHard)
  • Wire.onRequestData(handler) -> handler returns a byte to be sent to master. This byte can be generated from the slave in the handler at runtime without using the shared buffer. The official Wire does it transparently, retrieving the data from the buffer (not present in Wire, new in the WireHard)
  • Wire.onRequestDataNack(handler) -> handler takes no parameters. Used when the master still requires data which is not available from the shared buffer. It is done transparently in the official Wire (not present in Wire, new in the WireHard)

All the functions marked as new in the WireHard are the core difference between the official Wire library. The normal Wire handle them automatically, always reading or writing in the shared buffer, transparently to the user.

Initialization

Again, follow the official Wire guide. But this version supports a HARD_WIRE_MODE that shall be notified in the init. If mode is omitted, it is initialized following the normal Wire initialization and the additional handlers are still supported, but is not mandatory to use them. Actually, is never a mandatory to use the additional handlers.

  • Wire.begin(slaveAddress, HARD_WIRE_MODE)  -> will be supported all the additional handlers, with an undefined (vitually infinite) number of read/write cycles, as any I2C slave device would do.
  • Wire.begin(slaveAddress, NORMAL_MODE)  -> will be supported all the additional handlers, but when the shared buffer is full, it needs to be flushed or read before accepting any new incoming byte: this behavior is the same as the Wire library.

Hope this can be useful to someone as it was to me. More info and example on the GitHub page.

Download from GitHub.

Network programming – an introduction

As a support towards a complete tutorial and presentation of the “An UART interpreter“, I discovered the need to learn something about networking under Python language. In this way, I should be capable to enable the access to Internet of my microcontroller based board.

So I will start by showing an example on how to send any signal to the other end of a network – here in the same localhost, i.e. send messages from my PC to itself, but once the mechanism is set, we can speak to the other side of the planet by simply changing the IP address. By doing this, the UART interpreter will be IoT-enabled.

For everyone who don’t know anything about networking softwares and modules and wants to learn something, like me, the best thing is to start from the bottom layer of the network: sockets. Which are not so bottom, but let accept this trade off. In these examples is developed a sample software in Python 3 running on two machines, here used the same PC speaking in local, i.e. the address 127.0.0.1, or localhost. I used Python because is well suited to easily work with the RaspberryPi, which will be exploited soon in next projects. And can be used to test the set of networking scripts described here.

What is a socket? Basically it’s an “istance” of two point of communication between processes, or in other words are an IPC methodology (Inter-Process Communication). You can use them on local machine or between machines separated by any kind of networks. Of course, there are more efficients methods to allow two processes to communicate on the same machine, but sockets are almost the only simple solution to allow communication over the network. From the user point of view, the socket represent an IP address to which you want to connect to by using a specific “port” that can (but don’t have to) be different depending whether it is a request or an actual data response. A port then can be considered as an instance of communication with that IP, which can be treated in a different way as different as the port chosen, for that reason you should not use ports with a fixed function and causing possible conflicts between processes. Use for example:

127.0.0.1:8000

Thanks to sockets, you can also see this website, where data pass through your IP at port 80. For a complete description I invite you to search around, since explaining them deeply is not the purpose of this article and I’m not capable to.

Server side

In Python, we need to import the module socket in order to use a server socket and binding it at localhost address. This is the server address, but can be anything else than localhost (127.0.0.1). Then we listen for a certain number of connections at a time, here just one connection at a time is allowed, but typically is 5. The socket is an INET type  since we deal with internet protocol addresses and is STREAM because I want to keep monitoring of the activity until I decide to disconnect from it. If this is DGRAM, I send my data and then disconnect. But here I want to disconnect at the end of the client execution only.

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 8000))
server.listen(1)

Then in a loop we need to accept the connection, creating a client socket with its address, then receive from the client a string saved in mex and send the server aknowledge to the client.

while 1:
 #accept connections from outside
 print(&amp;quot;Server notification: waiting for client&amp;quot;)
 (clientsocket, address) = server.accept()
 mex = receive_sck(clientsocket)
 print(&amp;quot;Server notification: \n\tIncoming data: &amp;quot; + mex)

 msg = &amp;quot;Server notification: data received&amp;quot;
 msg = msg_len(msg)
 send_sck(clientsocket, msg)

A sad but true thing is that inside blocking sockets I can wait forever because there is not a prefixed size. So I’ve implemented by using the msg_len() method a protocol which attach before the message the number of chars followed by an underscore character. Of course, the number include the number of char used by the number it self, too. The method return the modified string.

In this way at receiving part I used the receive_sck()  to receive and check if in the received part the “_” is present, thanks to this I can identify the number. After that I set the MSGLEN to this number, exiting from the while after that amount of data is received, and if present, more data is discarded since is not part of the message, according to this protocol. Must be noticed that even if the maximum buffer allowed is 2048 chars, I can receive even only one char at a time, if the line is congested. So a marking char must be searched every time I receive from socket until I find it. The marking char here is “_”.

With the send_sck() is more simple, because what I send is known, but must be previously transformed to attach the number of chars, useful at the receiving part. When transmitting an error handling can be implemented.

Client side

On the other side of the connection the same methods are reused for modularity. They can be improved by using heavily a self created class, which I didn’t done here.

client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(('localhost', 8000))
msg = 'Hello IoT!'
msg = msg_len(msg)
send_sck(client, msg)
client.shutdown(1)
mex = receive_sck(client)
print(&amp;quot;Client notification: \n\tIncoming data: &amp;quot; + mex)
client.close()

Here I’ve implemented the socket client and connected it to server, the message msg is sent to the client socket called with fantasy client , sending the content to the server at localhost, i.e. the server address. The new thing is the client.shutdown(1), in which I close the client transmission capability allowing it to still receive, just to check the aknowledge from the server. Then I close the socket, disconnecting from server.

Conclusion

As a result, this is how two separate processes (server.py in the white backgroung and client.py executed on prompt of Windows) are communicatin each other. Client script is executed after server is waiting for client. Raw data received from each part is printed on stdout after the “Incoming data: ” and the actual payload contains the number of chars added by the protocol. These can be filtered out when printed, because are needed only by the algorithm and usually not by the user.

serverclient

Here you can dowload both client and server and play with them.

Source Code:
https://www.dropbox.com/s/766bokip1wogr0o/client.py?dl=0
https://www.dropbox.com/s/8fz2yz352r58ut1/server.py?dl=0