Before I can get to my main point, I would like to share some background with you.
For too many years, my life was focused (almost exclusively) on technology. Learning to code, spending hours upon hours online each day. Exploring, learning, feeding the never-ending hunger for more information because of my excitement when I first discovered some people in an online chat who happened to be writing code. They had the ability to think of something and then actually build it. I thought it was amazing, and it absorbed me.
Problem: when you do something too much, it’s too much.
And for a long time, I forgot how to do everything else. I forgot how to human.
As you might know, I freelance as a full stack developer (and software engineering), I build my own projects (startups) and I try to learn new technologies as often as I can. I try not to stand still. You would think that these things are relevant, or that they would somehow explain why/how things changed for me, but the truth is they’re not – and they don’t. But as I’ve come to notice, the most significant changes in my life have been due to the most absurd or unimaginable things possible.
However, I can describe the things that I do, the things that I focus on, and hope that it helps to explain who I was (or am) as a person when these things happen(ed). It’s the best I can do.
I can’t remember where I’ve first read this quote, but life is what happens while you are busy making other plans. (According to Wikipedia, it was Allen Saunders.)
And so I was confronted with how to human.
I met some people who are nothing like me.
I’m going to try to explain who they are – or at least the effect they’ve had on me – as best as I can, but written text isn’t the best at conveying emotion, so for the next part I hope you try to imagine it as if it would be happening to you. I hope you can open up.
They aren’t involved as much in technology, they aren’t focused on building commercial projects, we don’t work in the same fields and we’re not even from the same country. They just happen to be exchange students and they’re working to get their Master’s degree in human ecology. And I somehow just happened to meet them.
Even though I was an absolute stranger to them, they chose not to be a stranger to me. They chose to be the most honest, open, selfless and supportive people I’ve ever met or seen, more than I’ve ever seen anyone even try to be. If you would ask me, I would not be able to come up with any reason for why I would have “deserved” it or for why it ultimately happened. It just did.
Change, especially significant personal change, can be scary and overwhelming… and it was. It was so overwhelming that it hit me like a truck. It was so much, so fast.
But they didn’t mind, and they always stayed themselves.
I thought I was doing pretty well, taking care of myself. But meeting them made it clear to me that I was forgetting something important. I wasn’t standing still to seriously think about everything and everyone else. I was (unintentionally) selfish because of my inaction.
They taught me a lot. In so little time, they changed me.
One of the things they reminded me of, is that we should remember to care for each other more. We shouldn’t just close our eyes and act as if we don’t notice. We shouldn’t say we don’t care, just because it’s not happening to us personally. We shouldn’t distract ourselves with trivial things so we don’t have to look at the hard stuff.
Today I want to give back to them. They are the reason I’m sharing all this with you, and I’m hoping that we can make something greater together.
I want to show them that their efforts are worth something, and that everyone can change the world in their own way if they truly want to.
The birthday of one of them happens to be next Monday the 12th and I want to give this as a gift. The gift of letting them know they’re making real change, that they’re bringing positive change to a lot of lives. The gift of significant change and real impact.
And I need your help to do it.
I’m hosting a charity fundraiser for The Ocean Cleanup.
I want to give something back that helps people improve our world, to help build a better future.
In short, The Ocean Cleanup develops advanced technologies to rid the world’s oceans of plastic. There are over 5 trillion pieces of plastic currently littering the ocean. Ocean currents concentrate plastic in five areas in the world: the subtropical gyres, also known as the world’s “ocean garbage patches”. Once in these patches, the plastic will not go away by itself.
On May 11th, they unveiled the results of two years of work: an improvement to their design that will enable them to start extracting plastic within the next 12 months. With the first deployment in the Great Pacific Garbage Patch in the first half of 2018, they will start cleaning up two years ahead of schedule.
The Ocean Cleanup solves an important problem on a massive scale by intelligently using their technology. It’s a solution to a real problem that can be used in a matter of months.
From today on, we’re hosting a public fundraiser to gather as many donations as we can within this month (until the 30th).
To help The Ocean Cleanup, you can donate via this link: fundraiser has ended.
(We chose to use Paypal because other payment processors, including GoFundMe, take 4%+ of all donations.)
To start, I’ve donated € 350 myself and I’m adding more in a few days. This month, I will also give away 100% of contributions that I usually receive via Patreon.
Every day, I’ll update the total amount of raised donations at the top of this article.
I just hope that you’ll consider joining us in donating. Together, we can do more than when we are alone.
Please consider sharing the article (at the top of this article are some buttons!).
To anyone who has helped, from the bottom of my heart, thank you. ♥
Since my last article about things going slightly wrong (PR 1900), things have already improved. First off, an immense thank you to everyone who decided to support us. It’s thanks to your support that we can keep working on what we love, in the way that we do things best.
To jump right into it, here are the first two major points we’re already working on:
We’re replacing RocketMap’s Flask dev webserver (runs on Python).
We’re building an entirely new modular front-end that will replace all current webpages with a much faster alternative.
1. We’re replacing RocketMap’s Flask dev webserver.
Flask is a development webserver that runs on Python – it’s not fast, it’s not particularly good either, and it’s even recommended to use only for development and never for production. Using it during RocketMap’s development was fine, as it was only used by developers, but ever since a community was built around RocketMap that has changed and it’s time to move forward to a proper webserver.
We’re replacing Flask with a fully asynchronous Node.js webserver on Express.js that uses Handlebars for templating (with enabled view caching), Sequelize for ORM, and that supports gzip compression and load limiting with toobusy-js.
In short, your webserver will support much more concurrent connections (Node.js’ asynchronous design is perfect for it), it will be able to scale up properly (and can easily be used with load balancing behind nginx) and the load limiter will automatically refuse requests when it notices the load on your server is too high, so instead of queuing up more requests waiting to be handled and further increasing the load, it will handle the requests it can handle and tell the rest to try again later.
Using nginx as a reverse proxy for Flask already helped support more users, but this new webserver will give you a codebase that can support concurrency on a much bigger scale. You no longer need to fiddle around to “get it to work”: it will now work properly by default and you can further scale up/optimize with other technologies such as nginx reverse proxy w/ caching.
2. We’re building an entirely new modular front-end (with additional JS abstraction layer) that will replace all webpages.
Yup, it’s happening.
Gone bootstrap, gone the finicky switches and nearly non-existent mobile support, gone the never-ending lag when too many Pokémon are on the map.
With a focus on responsive design, mobile support, minimal file size and performance, we’ve changed the old way of developing the front-end. With an entire new stack (Bourbon, Neat, npm and other modules), we’re writing responsive SCSS code to convert to minimal CSS (the file size is a fraction of what it used to be).
And what does the “modular” in the title mean? Well, we’ve built several optional customizable blocks into the front-end code, meaning you get a few blocks in which you can put your own HTML/CSS/JS code without having to edit our files. You can customize the front-end while still being able to update your codebase (git pull) without conflicts. For now, there is a customizable header title & icon, two customizable header content blocks (top left and top right corners of the screen), and optional customizable menu items.
If you’re not sure what this all means, just know that it’ll support millions of active markers without lagging; you’ll be able to zoom out to see the world map without having to hide Pokémon markers. Even on mobile.
(While right now it can’t even support a few tens of thousands without lagging on a desktop…)
And as usual, there’s more coming.
The two points we’re working on are already massive changes compared to what has been changed in RocketMap’s history, and we’re going to need a fair bit of time to complete it, but these are the first steps towards an immensely more stable and more efficient RocketMap.
We have much more on our list of to-dos, some of which are smaller ones that we’ll work on during development of the above two points (e.g. the auto-verification mailserver we’ve recently released for all of our Patrons), and we’ll continue to keep you updated.
Remember that you can find us on our Discord, and that we’d love to hear your thoughts on these changes. ♥
By now, you know the daily routine: Niantic forces an API update, pgoapi gets updated, RocketMap implements the update. Or Niantic is silent for a while and RocketMap focuses on reworking old code, reviewing new code, and implementing new features. It sounds simple enough.
As my previous articles have explained, software development comes with its challenges, and open source adds some extra challenges. You might have noticed that things don’t always go smoothly: the merge of PR 1900 (SpeedScan improvements, by rocketrobot) and its unfortunate circumstances.
This article hopes to address these concerns and will explain our long term vision. Everyone currently in devkat has helped write and improve it, including the people you’ve known for a long time (Thunderfox, Dreggy, j4k3y and others).
For those who weren’t around, and for those who want an explanation.
RocketMap has a large workload but not enough people to finish everything instantly. This is, in itself, not an issue: having a work queue can actually be better than finishing everything instantly, as you can prepare a certain throughput of completed work per week rather than having moments where you’re just twiddling your thumbs. There’s always something interesting to work on.
For pull requests (open source work submitted by volunteers) to be accepted/merged, the only official requirement is two approvals on Github (preferably by a known #pr Discord member). Another requirement is to use common sense: there are often cases were two approvals are simply not enough.
Unfortunately, software code isn’t so simple that two people can click the “Approve” button and that we can expect everything to go perfectly for everyone if we decide to merge the changes. Computer Science is not something that can be based on “but it works for me!”, and this is often forgotten.
Which brings us back to PR 1900.
1900 was a large pull requests that didn’t do one thing, it tried to fix 12 separate issues in a single pull request. Common sense (or experience) would dictate to separate the pull request into 12 separate PRs and to review each one separately. Unfortunately, again, RocketMap is not an organization with official processes – until now, it never tried to be, in favor of being as open as it can be, meaning RocketMap was the result of all combined efforts put in by all the volunteers. A noble thought, but not always an efficient one.
Besides having a large PR to review, the “approvals” on Github started pouring in: people who simply pulled the changes into their local installation, who ran it, and then approved the PR because “hey, it works for me!”, entirely ignoring the fact that thousands of other people are running our code on different types of setups with different kinds of hardware and configurations, and ignoring code quality and efficiency.
I want to repeat it one last time: Computer Science is not based on “hey, it works for me!”. (And maintaining a project to uphold a certain level of quality is not “perfectionism”, it’s just not being lazy.)
But with the few volunteers working on RocketMap, we ran into an issue: we didn’t have people around that could go into detail to properly review PR 1900. I could, but besides the large amount of time that goes into RocketMap (for no real life return), I also have a life to maintain (Oh, and I just became an uncle! My sister just had a baby, Elias.).
With no other option, I offered to review PR 1900 in its entirety, with the immense side note that it would take a lot of time and that people would need to be patient.
Patience is however not a virtue because it’s always easy, and this was demonstrated in the days while I was reviewing PR 1900. Even the people who had been contributing to RocketMap and who knew inside & out what I was up to (including my holiday, its duration, and I was even online on Discord while on a mountain with my snowboard strapped to my feet in France), ended up PM’ing me and poking me to merge 1900, fully aware that it would take much more time (I repeated this often enough).
Which brought us to a point where one thing became obvious: people didn’t want a review, they wanted a merge.
Those who really wanted a review (which includes rocketrobot, the author of the PR), were immensely patient and understanding. I thank those people for that. But everyone else was so obsessed with getting it merged because they personally liked the PR and “hey, it worked for them!” that they lost sight of everything else.
So I merged PR 1900 without finishing my review.
One main reason: RocketMap is open source. As far as our development went, everyone should have been equal. But I realized that this wasn’t the case: when it came to taking responsibility and carrying workload on our shoulders, I realized that everyone simply expected me to continue being responsible for the majority of what got merged. And this is wrong: an Approval on Github brings responsibility with it – you can’t approve something without also saying that you’re giving your word that you’ve done everything necessary to properly review the code, and that you’re ready to be partially reponsible for the merge.
PR 1900 had 12 different people who approved the PR, two people who just discussed it and one who said not to merge (me). 12 people…
So I merged.
Of course things went wrong. While it “worked for some”, there were inevitable side effects (which we knew beforehand would happen), and for others it broke their map entirely. And so everyone was suddenly hit with reality.
There are some who have held me personally responsible and who are angered at me for what happened. I can understand why, but I also feel that those points of view are ignoring what really happened for weeks. I will say that it’s immensely unfortunate that we annoyed so many people, but I also won’t apologize for my decision. I explained my thinking, and I still stand behind it.
So it happened – now we learn and improve.
It’s frustrating that it happened.
So how do fix it for the future?
First, I believe that, if RocketMap has done one thing very right, it’s that it has always continued to struggle for what it believes is right. The staff, the volunteers, even the community, has occasionally struggled but it was always in order to eventually learn and to improve. In some contexts, a struggle is exactly the best situation you can have, because it means no one is controlling anyone and everyone has the opportunity to participate.
But for software development, a struggle leads to delays, and we want to fix this. We want RocketMap, in every sense of what RocketMap has become, to be an enjoyable experience.
So far, you’ve only occasionally heard of devkat. In simple terms, it’s the group of people that has been responsible for several of the responsibility, coding work, merges, and brainstorming contributions surrounding RocketMap. For the public, it has (seemingly) only been the group that hosts vote.devkat.org, because that has been the only time so far that we used the devkat name to publish something.
And devkat is how we want to solve the RocketMap problems: structure, official processes, sustainability. All of it, we want to tackle it.
We’ll work to:
Regularly brainstorm to improve how we work and what we work on.
Have a real development roadmap, structure and formal processes. Eventually we hope to have more frequent public communication (roadmap, milestones, todos, releases).
Contribute better and well-tested code to RocketMap.
Increase the amount of hours we can work on RocketMap. More support means we can trade in more of our regular working hours for RocketMap.
I’ve said it before: by separating devkat from RocketMap, RocketMap can continue to be what it is, freely. And devkat can continue to experiment without being constrained to what people have come to expect (even demand) from RocketMap.
We’ve been working on devkat for a few months now behind the scenes because we always wanted to create a development environment that works best for us. We wanted to solve some concrete issues that had already shown up, and we wanted to have a place and the people to solve issues that might pop up in the future. What happened with PR 1900 was a good reminder and a sign for us that it’s time to continue and to publish what we’ve been working on.
I’ll personally be working exclusively with devkat. I’ll still be around RocketMap as often as I used to, but all of my code will go towards devkat’s way of doing things – a proper way of working will benefit all of us. I feel that we can discuss for a long time how it’s – at least in some ways – unfortunate that money is what brings sustainability, but ultimately it’s the way the world works and I’d rather be realistic and adapt than hold an idealistic view that will eventually kill the project.
In a few concrete steps (explained in detail afterwards):
When we publish things made by devkat, we’ll officially announce it on devkat rather than silently contributing to RocketMap. This will help keep a clear focus on how we’ve been doing and what we can improve in devkat specifically.
The core group of devkat will forever stay a small group: growth is not our focus, code quality and a better RocketMap is. We’ll be accessible via our Discord server (http://discord.devkat.org). You can always apply if you feel it’s the right thing to do, but accepting or even replying is entirely at our own discretion – our focus comes first.
To solve the problem of not having formal processes, leading to untested/unreviewed code (e.g. PR 1900) or other roadblocks, devkat will host its own code on a private repository which we’ll maintain in a more formal way. There will be a way for contributors to formally contribute which will maintain our code quality, and there will be a way for interested people to participate in testing or using the code before it’s publicly released to RocketMap (more on this later). This repository will contain code at least two weeks before it gets released publicly to RocketMap.
Finished contributions to the private devkat repository will be tested and reviewed in private for at least two weeks before releasing the code to the main RocketMap repository (per pull request, as usual). Because of how we’ll work, we’re more than ready to take full responsibility for any devkat merges (so none of that PR 1900 again).
When available, we’ll provide paid hourly support for everything related to devkat code, setting up RocketMap, 3rd party tools (e.g. account gens) and unreleased/unmerged code (e.g. pull requests or devkat’s experimental code), from A to Z, including debugging and accessing private/custom setups. A quote can always be requested.
RocketMap itself will stay exactly as it is. It’s still an open organization, it still stays on Github, it still stays free (and remember: devkat releases for free to RocketMap two weeks after it publishes to its private repositories). If you want a more formal explanation, devkat is a separation of concerns and responsibilities, to allow devkat to experiment without having to affect what RocketMap is as an organization and community.
#3 and #4: devkat Patreon, Roles and Private Repository
devkat’s private repository and closed group will maintain a specific level of code quality. Stricter rules and development processes, things that can’t (or that we don’t always want to) expect or enforce in an open source project like RocketMap. We want to try to make RocketMap better by doing what we do best, and we want to do this by creating a sustainable, better environment for our developers, which will let us contribute better code to RocketMap.
Via Patreon pledges (devkat Patreon profile), you can financially support our efforts and get read access to the private repository (more details in the pledge options on Patreon), and we use Paypal for the private support (we set up a unique invoice for each request, just PM us on devkat’s Discord or send an e-mail to firstname.lastname@example.org).
1. To solve the problem of sustainability, devkat will offer paid read access to its private repository.
Some of our Patreon pledges includes read access to our private repository. By contributing, you get instant access to all finished code merges at least two weeks before it gets released publicly to RocketMap. Our higher Patreon contributors who want to support us more can also get access to our experimental code – this is similar to Github’s “pull requests”, except that it’ll be access to exclusively devkat code before it gets released to the devkat private repository for basic pledgers.
After you’ve pledged and you’ve received your Discord rank, just PM us on Discord or send an e-mail to email@example.com to request access to the repository. We only send your login details after you’ve requested them, so make sure to let us know you want access.
2. We’ll provide paid hourly support for RocketMap, devkat, and 3rd party tools.
As another way to support the development, some of our members will be open to setting things up for you when it gets too difficult. Whether it’s the basics of getting set up, helping you purchase whatever is required (proxies, hashing keys, servers) or helping you debug your custom installation/configuration, we can help you out.
If you need our help, PM us on Discord or send an e-mail to firstname.lastname@example.org. We’ll give you a quote, and all invoices will be sent via Paypal (We’re a registered company, we pay our taxes. It’s all good. ♥).
3. Volunteer contributors can earn bounties by contributing to devkat.
To motivate the developers who take the time to do things properly, we’re going to work with a bounty system. When you, as a developer, want to write code for RocketMap, we want to give you two options:
You release to RocketMap as usual and go through the old process.
You contact us via devkat’s Discord and you release to devkat’s private repository. Your code will go devkat’s usual process of review by its members, followed by a thorough testing process, on to the paid read-only repository for our supportive community members. Two weeks later, it’s at its final stage and has been properly reviewed, after which it gets merged publicly into RocketMap.
If you choose to work the devkat way, you’ll earn a bounty for your work. All bounties depend on the type and volume of your work, and everything surrounding the bounties is communicated before you submit anything to devkat. No surprises. We also welcome any volunteers who simply want to contribute code to be reviewed by devkat members.
PR contributors who have an open PR and want to move it to devkat before it gets merged can do so by PM’ing us on Discord. Just reminder that it’s first come first served, we can’t (and won’t) take up more work than we can handle.
As an important side note, contributing code does not require you to have a paid Patreon pledge. We received a question about this, and we hope this clears it up.
4. And once we get the ball rolling, we invest more into RocketMap.
Public projects, hosting servers, distributing data, investing in our own RE team for a free API, writing dynamic/polymorphic systems to permanently block scrapers, … everything we can try, we will try.
RocketMap is what we’ve always been and what we’ll always be about. We’ll continue to work on improving RocketMap and on contributing new items that will make it easier to set up and use, so we can get back to a time where people used the maps to play together as groups of friends and family. Only this time, more professionally.
It’s time to make RocketMap great again.
We’re not going to build any walls (at least not unless they make sense for RocketMap), but I believe we can use a bit of their optimism to work towards a more positive message: making RocketMap great(er).
Continuously experimenting and trying new things to learn from, so we can see for ourselves what really works.
Thank you for your continued support. ♥
Today I don’t leave you with a song but with a message.