How to get Threema end to end gateway working on a raspberry pi
Since documentation is very limited, and I just successfully did this, I decided to write down how I did this myself. In this howto, I share how I managed to send and receive messages using the Threema Gateway and my mobile phone. Messages are end-to-end encrypted.
Material needed
In order to do it the way I did it, you’ll need the following:
A linux computer which has access to the internet
I got it working on a raspberry pi 5 running PiVPN, PiHole, a telegram listener and this listener simultaneously.
It needs an open port to the internet being able to accept connections. I used 8443 since it’s also used in the python script.
It also needs port 80 to be open to the internet, this is required for the check on the TLS certificate when you generate one. After this howto, you can close port 80 again.
Remember that both ports need to be able to receive connections from the internet. Just opening them on your server might not be enough. If you’re behind a NAT router, the ports need to be forwarded there as well.
A smartphone to install the Threema app and send/receive messages.
About 60 euros on a payment method accepted by Threema, which is a bank transfer (takes time), a Visa or Mastercard or Bitcoin. 3 euros in credit in your app store or play store to buy the app.
A few hours of handling time, a few days of waiting time
Step 1: get a gateway account
Pretty easily, go to https://threema.ch/en/gateway/products and register your account using the button on the top right.
It’s just a few details and an link in your e-mail to make sure it’s yours:
For the end to end option, you need at least 1600 credits, the closest is 2500 but you can buy more if you’d like. I advise to go for the 2500 for testing, and then take it from there. So on the threema website, buy 2500 credits (about 50 euros worth). Using your account. This can be done on the balance page,
Get the keypair to go with your account
Threema is nog going to make your public/private key pair for you, you need to do it yourself. Fortunately, there’s a handy guide to do that which you can find here: https://gateway.threema.ch/en/developer/howto/create-keys/java. I used the Java option and just found out there’s a Python option as well which probably works similarly. Just go to the site and click the blue button to download the relevant API:
Then unzip the file (you can use the GUI if you want to):
unzip threema-msgapi-sdk-java-2.1.0.zip
And then execute the command to generate a public and private key:
java -jar threema-msgapi-tool.jar -g privateKey.txt publicKey.txt
You might value reading what’s in the yellow warning box as well. You can generate keys as many as you like, but once you submit the public key to Threema it will be permanently linked to that ID. Lose either key and you’ll lose the accountThreema cannot regenerate them for you and neither can you yourself.
Get the gateway ID
Go back to your dashboard and request another Threema ID by clicking the blue button in the manage IDs area. If this is your first ID, this page might look slightly different.
A new form opens where you need to fill in some details. Note that, your Custom ID and your Public Key CANNOT be changed afterwards. Short explanation of the fields:
Your Custom ID - The ID will be shown in users contact list and needs to be exactly 7 characters. IDs will start with a Star (*)
Public Nickname - Will be shown in push notifications and can be longer, so usually a more recognisable name.
Public Key - The public key of the keypair you generated in the previous step. I included the starting “public:” but the website automatically removed it. Make sure there’s just the public key in there, and no spaces or other characters
URL for incoming messages - When Threema receives a message, where should it send it to? This is your listening domain and port we are going to set up later. I used a domain name and didn’t test it with an IP address, but my guess would be that that works as well. If you really want a domain name, you could use something like duckdns but that goes beyond the scope of this howto. You can change this later, so feel free to leave it empty for now.
Use - New requests get reviewed and accepted or banned based on their use. So think of what you want to use your Threema ID for and fill that in here. I filled something in the direction of that it’s for testing, developing and using for my personal notifications and productivity.
Approving this ID request is a manual process and Threema states it takes a few days. Mine took less than 24 hours, where about 12 of those were on a Sunday. So that’s quite fast, but if there are many requests, it could take a while.
Step 2: Get the software
On your phone, download the Threema app and create an account. The app costs 3 euros and can be used to message to others than just your own gateway. Be aware that this is a different account, with a different name, than the one you used to order your gateway. It is meant to be this way.
Unfortunately, I don’t have screenshots of the installation and configuration of the app, but it also was quite straightforward.
Certificates
They keys identify you with Threema, but don’t encrypt your connection with them. Therefore, you need a TLS connection to encrypt your connection. For a TLS connection, you need certificates.
I used to think that there’s no worse form of hell like certificates and encryption, but fortunately, there’s certbot. At this point, you should have your domain name ready, since that’s what you’re certifying here. Your domain should also point to the server you’re going to use for Threema and where you’re running these commands.
Start by installing the certbot:
sudo apt-get install certbot
And then just run it and generate your certificates:
sudo certbot certonly --standalone -d <yourdomain>
It answers some questions like who you are and where you’re from. Fine to answer, at some point it’ll generate certificats for you. If certbot refuses, the error message is quite clear on what to do. This is also the point at which you’ll need port 80 opened to the outside world, since certbot will use that to connect to and make sure it’s on the domain name you specified.
Make sure the key and certificate are on your server and your user has access to them. They are usually owned by root, so do some chmod or chown to get that fixed.
If you want a different CA or self signed, go ahead and figure it out yourself. You need both a certificate and a private key document.
Python dependencies
Don’t do everything yourself, but install some Threema packages to make your life easier. Usually Python packages are managed by either pip, which is python specific, or apt, which applies to the entire system. I couldn’t find the apt-get packages and was to lazy to look them up, so I foced pip to do it:
pip3 install threema.gateway --break-system-packages
You can probably do without the --break-system-packages, if you use pip to install your python packages anyway.
Get writing some code
Threema has git repositories with examples. I used theirs to get started with a script to send and receive messages which you can find here: https://github.com/threema-ch/threema-msgapi-sdk-python. Specifically, the callback example they have here: https://github.com/threema-ch/threema-msgapi-sdk-python/blob/master/examples/callback.py. The other files in the same folder are especially useful if you want to send/receive different types of messages.
It’s easy do just copy the file and save it as a .py file on your server, but it does need some adjustments. Here’s what I changed:
On line 22, there’s a to_id which is ECHOECHO, which just sends messages back. If you want it to be your phone, you need to change that here to the ID of Threema you created on your phone.
to_id='ECHOECHO',
On line 31, 32 and 33 you need to enter some credentials of the Gateway ID you have.
identity='*YOUR_GATEWAY_THREEMA_ID',
secret='YOUR_GATEWAY_THREEMA_ID_SECRET',
key='private:YOUR_PRIVATE_KEY'
The first one is your ID you gave in when you bought your gateway ID. The second one is a secret generated once your request has been approved. The last one is the private key of the keypair you generated and then gave the public key in the Threema form.
Finally, you need the TLS connection set up by fixing this line:
ssl_context.load_cert_chain(certfile='YOUR_CERTFILE', keyfile='YOUR_KEYFILE')
There, your certfile should point to the fullchain.pem, and the keyfile to the private key you generated.
After that, you can start your server and you should receive a message on your phone. You can send a message to your server after which it is printed in your terminal.
python3 ./callback.py
Troubleshoot
My biggest issue was with getting the connection and the certificates right. If you send a message to the server and nothing happens, you can use your browser and serve to the domain and port you set up. You should get a plethora of error messages saying it doesn’t understand but at least proofing the connection is open.
If you managed to generate the error messages when you use your browser, make sure the IP/domain you go to is exactly the same as what you configured in the gateway. If you’re sure, it could be that the certificates are wrong or not signed properly causing them to generate an error. So look into that.
Next steps
For myself, I’m not ready yet. For some things related to productivity I use Telegram and I want Threema to take over, so that means integrating the callback script we have at the end of this howto into my python code I already have. There are two things I found which I still need to resolve.
The first is that when receiving a message, the Threema library returns a TextMessage object from which I still need to extract the text. This seems minor, but it also involves digging up the documentation for the Threema Python library which I haven’t found yet.
The second is a bit more worrisome, which is that when the Threema listener is killed, just the python is killed but the interface stays open (a little headless, if you will). This is an obvious security issue, but it also prevents me from starting Threema again after shutting it down. I could restart the server (which is what I do now) but that is both cumbersome and there’s more running on there. So I’d rather not.
If you’re interested, let me know and I can write you another blog maybe (or go back to writing about Scrum and Agile, which is more my cup of tea… ;-) )