By now, you know the daily routine: Niantic forces an API update, pgoapi gets updated, RocketMap implements the update. Or Niantic is silent for a while and RocketMap focuses on reworking old code, reviewing new code, and implementing new features. It sounds simple enough.
As my previous articles have explained, software development comes with its challenges, and open source adds some extra challenges. You might have noticed that things don’t always go smoothly: the merge of PR 1900 (SpeedScan improvements, by rocketrobot) and its unfortunate circumstances.
This article hopes to address these concerns and will explain our long term vision. Everyone currently in devkat has helped write and improve it, including the people you’ve known for a long time (Thunderfox, Dreggy, j4k3y and others).
For those who weren’t around, and for those who want an explanation.
RocketMap has a large workload but not enough people to finish everything instantly. This is, in itself, not an issue: having a work queue can actually be better than finishing everything instantly, as you can prepare a certain throughput of completed work per week rather than having moments where you’re just twiddling your thumbs. There’s always something interesting to work on.
For pull requests (open source work submitted by volunteers) to be accepted/merged, the only official requirement is two approvals on Github (preferably by a known #pr Discord member). Another requirement is to use common sense: there are often cases were two approvals are simply not enough.
Unfortunately, software code isn’t so simple that two people can click the “Approve” button and that we can expect everything to go perfectly for everyone if we decide to merge the changes. Computer Science is not something that can be based on “but it works for me!”, and this is often forgotten.
Which brings us back to PR 1900.
1900 was a large pull requests that didn’t do one thing, it tried to fix 12 separate issues in a single pull request. Common sense (or experience) would dictate to separate the pull request into 12 separate PRs and to review each one separately. Unfortunately, again, RocketMap is not an organization with official processes – until now, it never tried to be, in favor of being as open as it can be, meaning RocketMap was the result of all combined efforts put in by all the volunteers. A noble thought, but not always an efficient one.
Besides having a large PR to review, the “approvals” on Github started pouring in: people who simply pulled the changes into their local installation, who ran it, and then approved the PR because “hey, it works for me!”, entirely ignoring the fact that thousands of other people are running our code on different types of setups with different kinds of hardware and configurations, and ignoring code quality and efficiency.
I want to repeat it one last time: Computer Science is not based on “hey, it works for me!”. (And maintaining a project to uphold a certain level of quality is not “perfectionism”, it’s just not being lazy.)
But with the few volunteers working on RocketMap, we ran into an issue: we didn’t have people around that could go into detail to properly review PR 1900. I could, but besides the large amount of time that goes into RocketMap (for no real life return), I also have a life to maintain (Oh, and I just became an uncle! My sister just had a baby, Elias.).
With no other option, I offered to review PR 1900 in its entirety, with the immense side note that it would take a lot of time and that people would need to be patient.
Patience is however not a virtue because it’s always easy, and this was demonstrated in the days while I was reviewing PR 1900. Even the people who had been contributing to RocketMap and who knew inside & out what I was up to (including my holiday, its duration, and I was even online on Discord while on a mountain with my snowboard strapped to my feet in France), ended up PM’ing me and poking me to merge 1900, fully aware that it would take much more time (I repeated this often enough).
Which brought us to a point where one thing became obvious: people didn’t want a review, they wanted a merge.
Those who really wanted a review (which includes rocketrobot, the author of the PR), were immensely patient and understanding. I thank those people for that. But everyone else was so obsessed with getting it merged because they personally liked the PR and “hey, it worked for them!” that they lost sight of everything else.
So I merged PR 1900 without finishing my review.
One main reason: RocketMap is open source. As far as our development went, everyone should have been equal. But I realized that this wasn’t the case: when it came to taking responsibility and carrying workload on our shoulders, I realized that everyone simply expected me to continue being responsible for the majority of what got merged. And this is wrong: an Approval on Github brings responsibility with it – you can’t approve something without also saying that you’re giving your word that you’ve done everything necessary to properly review the code, and that you’re ready to be partially reponsible for the merge.
PR 1900 had 12 different people who approved the PR, two people who just discussed it and one who said not to merge (me). 12 people…
So I merged.
Of course things went wrong. While it “worked for some”, there were inevitable side effects (which we knew beforehand would happen), and for others it broke their map entirely. And so everyone was suddenly hit with reality.
There are some who have held me personally responsible and who are angered at me for what happened. I can understand why, but I also feel that those points of view are ignoring what really happened for weeks. I will say that it’s immensely unfortunate that we annoyed so many people, but I also won’t apologize for my decision. I explained my thinking, and I still stand behind it.
So it happened – now we learn and improve.
It’s frustrating that it happened.
So how do fix it for the future?
First, I believe that, if RocketMap has done one thing very right, it’s that it has always continued to struggle for what it believes is right. The staff, the volunteers, even the community, has occasionally struggled but it was always in order to eventually learn and to improve. In some contexts, a struggle is exactly the best situation you can have, because it means no one is controlling anyone and everyone has the opportunity to participate.
But for software development, a struggle leads to delays, and we want to fix this. We want RocketMap, in every sense of what RocketMap has become, to be an enjoyable experience.
So far, you’ve only occasionally heard of devkat. In simple terms, it’s the group of people that has been responsible for several of the responsibility, coding work, merges, and brainstorming contributions surrounding RocketMap. For the public, it has (seemingly) only been the group that hosts vote.devkat.org, because that has been the only time so far that we used the devkat name to publish something.
And devkat is how we want to solve the RocketMap problems: structure, official processes, sustainability. All of it, we want to tackle it.
We’ll work to:
- Regularly brainstorm to improve how we work and what we work on.
- Have a real development roadmap, structure and formal processes. Eventually we hope to have more frequent public communication (roadmap, milestones, todos, releases).
- Contribute better and well-tested code to RocketMap.
- Increase the amount of hours we can work on RocketMap. More support means we can trade in more of our regular working hours for RocketMap.
I’ve said it before: by separating devkat from RocketMap, RocketMap can continue to be what it is, freely. And devkat can continue to experiment without being constrained to what people have come to expect (even demand) from RocketMap.
We’ve been working on devkat for a few months now behind the scenes because we always wanted to create a development environment that works best for us. We wanted to solve some concrete issues that had already shown up, and we wanted to have a place and the people to solve issues that might pop up in the future. What happened with PR 1900 was a good reminder and a sign for us that it’s time to continue and to publish what we’ve been working on.
I’ll personally be working exclusively with devkat. I’ll still be around RocketMap as often as I used to, but all of my code will go towards devkat’s way of doing things – a proper way of working will benefit all of us. I feel that we can discuss for a long time how it’s – at least in some ways – unfortunate that money is what brings sustainability, but ultimately it’s the way the world works and I’d rather be realistic and adapt than hold an idealistic view that will eventually kill the project.
In a few concrete steps (explained in detail afterwards):
- When we publish things made by devkat, we’ll officially announce it on devkat rather than silently contributing to RocketMap. This will help keep a clear focus on how we’ve been doing and what we can improve in devkat specifically.
- The core group of devkat will forever stay a small group: growth is not our focus, code quality and a better RocketMap is. We’ll be accessible via our Discord server (http://discord.devkat.org). You can always apply if you feel it’s the right thing to do, but accepting or even replying is entirely at our own discretion – our focus comes first.
- To solve the problem of not having formal processes, leading to untested/unreviewed code (e.g. PR 1900) or other roadblocks, devkat will host its own code on a private repository which we’ll maintain in a more formal way. There will be a way for contributors to formally contribute which will maintain our code quality, and there will be a way for interested people to participate in testing or using the code before it’s publicly released to RocketMap (more on this later). This repository will contain code at least two weeks before it gets released publicly to RocketMap.
- Finished contributions to the private devkat repository will be tested and reviewed in private for at least two weeks before releasing the code to the main RocketMap repository (per pull request, as usual). Because of how we’ll work, we’re more than ready to take full responsibility for any devkat merges (so none of that PR 1900 again).
- When available, we’ll provide paid hourly support for everything related to devkat code, setting up RocketMap, 3rd party tools (e.g. account gens) and unreleased/unmerged code (e.g. pull requests or devkat’s experimental code), from A to Z, including debugging and accessing private/custom setups. A quote can always be requested.
RocketMap itself will stay exactly as it is. It’s still an open organization, it still stays on Github, it still stays free (and remember: devkat releases for free to RocketMap two weeks after it publishes to its private repositories). If you want a more formal explanation, devkat is a separation of concerns and responsibilities, to allow devkat to experiment without having to affect what RocketMap is as an organization and community.
#3 and #4: devkat Patreon, Roles and Private Repository
devkat’s private repository and closed group will maintain a specific level of code quality. Stricter rules and development processes, things that can’t (or that we don’t always want to) expect or enforce in an open source project like RocketMap. We want to try to make RocketMap better by doing what we do best, and we want to do this by creating a sustainable, better environment for our developers, which will let us contribute better code to RocketMap.
Via Patreon pledges (devkat Patreon profile), you can financially support our efforts and get read access to the private repository (more details in the pledge options on Patreon), and we use Paypal for the private support (we set up a unique invoice for each request, just PM us on devkat’s Discord or send an e-mail to email@example.com).
1. To solve the problem of sustainability, devkat will offer paid read access to its private repository.
Some of our Patreon pledges includes read access to our private repository. By contributing, you get instant access to all finished code merges at least two weeks before it gets released publicly to RocketMap. Our higher Patreon contributors who want to support us more can also get access to our experimental code – this is similar to Github’s “pull requests”, except that it’ll be access to exclusively devkat code before it gets released to the devkat private repository for basic pledgers.
After you’ve pledged and you’ve received your Discord rank, just PM us on Discord or send an e-mail to firstname.lastname@example.org to request access to the repository. We only send your login details after you’ve requested them, so make sure to let us know you want access.
2. We’ll provide paid hourly support for RocketMap, devkat, and 3rd party tools.
As another way to support the development, some of our members will be open to setting things up for you when it gets too difficult. Whether it’s the basics of getting set up, helping you purchase whatever is required (proxies, hashing keys, servers) or helping you debug your custom installation/configuration, we can help you out.
If you need our help, PM us on Discord or send an e-mail to email@example.com. We’ll give you a quote, and all invoices will be sent via Paypal (We’re a registered company, we pay our taxes. It’s all good. ♥).
3. Volunteer contributors can earn bounties by contributing to devkat.
To motivate the developers who take the time to do things properly, we’re going to work with a bounty system. When you, as a developer, want to write code for RocketMap, we want to give you two options:
- You release to RocketMap as usual and go through the old process.
- You contact us via devkat’s Discord and you release to devkat’s private repository. Your code will go devkat’s usual process of review by its members, followed by a thorough testing process, on to the paid read-only repository for our supportive community members. Two weeks later, it’s at its final stage and has been properly reviewed, after which it gets merged publicly into RocketMap.
If you choose to work the devkat way, you’ll earn a bounty for your work. All bounties depend on the type and volume of your work, and everything surrounding the bounties is communicated before you submit anything to devkat. No surprises. We also welcome any volunteers who simply want to contribute code to be reviewed by devkat members.
PR contributors who have an open PR and want to move it to devkat before it gets merged can do so by PM’ing us on Discord. Just reminder that it’s first come first served, we can’t (and won’t) take up more work than we can handle.
As an important side note, contributing code does not require you to have a paid Patreon pledge. We received a question about this, and we hope this clears it up.
4. And once we get the ball rolling, we invest more into RocketMap.
Public projects, hosting servers, distributing data, investing in our own RE team for a free API, writing dynamic/polymorphic systems to permanently block scrapers, … everything we can try, we will try.
RocketMap is what we’ve always been and what we’ll always be about. We’ll continue to work on improving RocketMap and on contributing new items that will make it easier to set up and use, so we can get back to a time where people used the maps to play together as groups of friends and family. Only this time, more professionally.
It’s time to make RocketMap great again.
We’re not going to build any walls (at least not unless they make sense for RocketMap), but I believe we can use a bit of their optimism to work towards a more positive message: making RocketMap great(er).
Continuously experimenting and trying new things to learn from, so we can see for ourselves what really works.
Thank you for your continued support. ♥
Today I don’t leave you with a song but with a message.