Gomobile-IPFS
It has been almost a year since the lightbulb went off: “Hey! Everyone! We’re making gomobile + IPFS happen!”
Why the urgency? Read on to learn about the inception of the gomobile-ipfs project, its evolution, and some lessons learned along the way.
The birth of gomobile-ipfs idea 💡
During the development of Berty v1 (2018-2019), we were using LibP2P : the underlying IPFS library provided by the Protocol Labs team to easily develop distributed applications. At the time, we chose to develop a custom Berty node that was not fully compatible with a standard IPFS node.
More info about this technical choice in this blog post: How Berty Works
In June 2019, we met the IPFS community during our stay at IPFS Camp , a hacker retreat organized by Protocol Labs . It was an awesome opportunity to meet a experts in the field and banter about problems and solutions in our technical and philosophical roadmap.
We also met the team behind the famous Textile project, which provides an open-source set of tools that facilitate the development of distributed mobile and desktop applications over IPFS. They took the initiative to organize workshops, deep dives and brainstorming sessions.
One such session was on the theme “IPFS on mobile”, so we were keen to participate given our extensive experience on the subject. After a fruitful discussion with Carson Farmer , Core Developer @Textile.io, we realized that we were encountering the same problems with using IPFS on mobile:
- Most cellular connections are too unreliable and slow, and make NAT traversal techniques ineffective. The only one we’ve been able to get to work reliably is libp2p’s TURN-like protocol (relay).
- When switching between a cellular and wifi network, all connections are closed (except in the case of QUIC transport).
- Computing resources (CPU, battery, network) are relatively limited on mobile devices. Opening connections with several hundred peers has a huge impact on a smartphone (even a high-end ones).
- Smartphone OSes show no mercy for application lifecycle duration. After a few seconds of our app running in the background, the IPFS node is killed. This implies that it must be restarted when the app returns to foreground and that the tens of seconds needed for the node to reconnect to a significant number of peers makes the use of the application very painful.
Seeing the enormous potential of using IPFS on mobile (but well aware of its challenges), we brought up the idea of launching the gomobile-ipfs project with interested community members.
First PoC
On our return from IPFS Camp, we concluded that it was time to drop our v1 and start working on gomobile-ipfs as the foundation for v2, while making it usable for anyone.
The additional benefit of developing a more generic tool was to bring our work to the community and benefit from their contributions in return.
We started by making a simple PoC displaying the IPFS WebUI using React-Native for the UI and a go-ipfs node compiled with gomobile.
After we hit this milestone (🥳), we started to think about how best to bind the full functionality of the go-ipfs node with three bridge packages that could be used by mobile developers:
- Java for Android
- Swift for iOS
- React-Native for iOS/Android
The architecture of gomobile-ipfs
During a few architecture sessions on our side and some exchanges with the Textile teams, we had considered several solutions to bind the coreAPI functions between the golang package and the native packages:
- Manually via gomobile
- Using Protobuf and gRPC
- Using the existing HTTP API
The main advantage of using the HTTP API compared to the other two solutions is that it meant much less work for the initial implementation and to maintain existing functionality as the coreAPI evolved.
Another advantage is that complete documentation already exists for the HTTP API. A developer using it in pre-existing code will be able to port their application more easily.
Finally, we built out the specifics of our plan: The go-ipfs node would serve its HTTP API on a Unix Domain Socket and the native packages would access it via the RequestBuilder of the shell package .
You can find the entire exchange on this issue .
Initial version
We quickly implemented a first version of gomobile-ipfs with the help of Steven Allen from Protocol Labs who reviewed our code in detail and helped us to fix a number of problems.
Then our repo was quickly moved to IPFS Shipyard and made open-source. It currently contains the following components:
- A golang part which exposes the essential functions for the init of the node and the functions related to the RequestBuilder
- Two Swift and Java packages that facilitate node management and send requests from these native languages
- Two demo applications for Android and iOS using gomobile-ipfs that simply download a random XKCD over IPFS
Since then, we’ve also set up a CI that publishes the different Maven packages (Android), Cocoapods packages (iOS) and the demo applications on Bintray .
The curious among you can already start using them in your project. A minimal doc is available here on the project’s Github Pages .
You can also find here a video that summarizes the basics of what there is to know: IPFS Weekly Call - 04/27 .
Next steps
There are still several points necessary for the project to be mature, some of which are already in progress.
Let’s take a look at our to-do list!
- Implement the lifecycle management of the node via iOS and Android APIs. Namely: manage the state of the node according to the state of the application (foreground, background, etc…)
- Make it possible to interact with the node through Streams rather than byte arrays. Gomobile by default only supports simple types , but it would be pretty useful to be able to use Streams to manipulate and access large files (to read the node’s logs, for example).
- Add the possibility to establish a direct connection with another peer and to implement handlers for a given protocolID. This boils down to to binding the
SetStreamHandler(protocolID, handler)andNewStream(peerID, protocolID)functions. - Further optimize the mobile node. This “step” will undoubtedly be more of a hike, and will require the implementation of advanced performance testing systems.
Finally, we are working on the development of drivers to establish direct BLE and Wifi connections, which will be directly integrated into gomobile-ipfs.
You can find more info on this subject by reading this blog post .
If you would like to help out, we’d be thrilled to get assistance from our community !
