Going to be really honest here. This wasn’t the easiest week for me. Didn’t make it to the launch like I wanted. Also had the blog for Build In Public University crap out on me and I lost the posts I had on there.
So let’s dig into the issues one by one.
First, I’ll outline the issue with the blog, because it’s a good learning experience. I’d decided to run a self-hosted Ghost instance on Render. Seemed easy enough to set up, I just followed the instructions and configured everything. A Ghost webservice and a MySQL database.
But then there was a blip in the service. Specifically, an AWS issue in the Ohio region. Downtime of about 5 minutes, and then came back up. The database went down. And later that day, I went to visit the blog because I was planning on grabbing some info from it and saw the default Ghost display. Imagine my surprise. I put in a support request and started trying to figure out if I could recover anything.
Thanks to a suggestion from Twitter, I checked out the Wayback Machine and was able to recover a single post, the first. The next day, I finally received an email from the support team at Render and found out that I had messed up in the config. I didn’t configure a disk to persist data between reboots. I would have been fine if I had, but I didn’t. And as such, my data was lost. I thought I had followed the steps perfectly, but alas, I did not. Thanks ADHD!
So, that’s something to remember. If you are configuring a database, it’s important to make sure that it can reboot without losing data. If not, you’ll be a bit sad when it happens and you didn’t expect it. Apparently, this is distinct to MySQL, because I didn’t need to set it up this way with my Postgres instance I setup on Render. I’ve got backups created periodically. The more you know, eh?
So, I ended up switching to a Ghost Pro account through the official site for now. I might end up going back to self-hosted if/when it grows enough to make it financially make sense, but for now, I can afford the $11/month for the minimum service to get it up and running and not have to worry about making sure everything is set up properly.
Product Launch Status
If you’ve been reading this newsletter for any amount of time, you’ve probably seen my struggles with Twitter’s OAuth. I’ve had to jump through all sorts of hoops to figure out how to make everything work the way I wanted.
So it may or may not be a surprise to you when I tell you that I started to implement the “unfollow all” feature that requires a long running process, I again ran into issues.
At first, I thought it was something I was doing. If you try to unfollow over 50 accounts, you hit the rate limits. So I had it set up to automatically try to refresh the OAuth token if it was older than 90 minutes, because the token is set to last for 2 hours, so I wanted to give it a little bit of a buffer. I worked on getting the bulk unfollow functionality added by testing with 5 accounts or so at a time, and then once I had the flow working the way I wanted, I kicked off the process for all the dormant accounts I followed and went to bed.
Imagine my surprise when I woke up and the count was no lower than it had been when I went to bed. Turns out, I was getting 401 Forbidden errors. So I started digging into it and found out that I was not the only one. I thought I had it figured out though: I kicked off multiple processes that could have potentially tried to refresh over and over at the same time, so maybe that was the issue. Time to refactor.
So I changed it to where there was 1 set task that was scheduled to run periodically and refresh all the tokens in the database, one at a time. I thought that would fix the problem. So again, I set all this up, kicked off the process, and went to bed.
Turns out, nope. That did not in fact fix the problem. It ran into the same issue. And apparently, there were a number of people running into this issue. And I didn’t see a satisfactory response from Twitter and it had seemingly been happening for months.
Back to the drawing board.
Initially, I had started with OAuth2 because of the limited access I could get with the developer account. You have to get elevated access which involves a review process to get Oauth1.1 access, which I didn’t do initially.
But I do have the elevated level of access now for this account. And there are Oauth1.1 endpoints for unfollowing, and lookups, and everything I need to do.
So I ended up spending an evening doing some refactoring and changing everything around to use the older OAuth implementation. I also made sure that I could easily switch between the two in the future.
I’ve got it set up to support multiple clients via configuration in a database table, so I ended up just adding a column to the table with the OAuth version to use. Then there is a check before going out to the Twitter API based on the config, and it pulls the correct keys based on that config and returns the data. That way, I don’t have to worry about it anymore except in that one spot.
I’ve actually been pretty happy with the way my codebase is evolving. Pretty much, every time I start to have issues with tracing the code around, I refactor and clean up. And as a result, the code has gotten a lot easier to manage. I look for things I do over and over again, and then pull that into a single location. So instead of doing a client check for each function that pulls data from the Twitter API, I’ve got a function that can determine the right client for a given account and return it to the function making the call to the Twitter API.
I also added a service layer between the Twitter API class and the tasks/views that I use. This way, I can have a layer that can make sure that I have the objects I expect and create new ones as needed and I don’t need to clutter up the top layer code with those kinds of checks/calls. As a result, most of my tasks and views have fewer than 10 lines of code. It’s much easier for me to figure out what is happening and where now, which increases my coding velocity. Highly recommend cleaning as you go. If you have trouble figuring out what’s going on in the code you wrote, that’s a good sign that it’s time to refactor and clean up.
And now, the functionality works. I was finally able to set it and forget it over night, and was able to unfollow a total of about 800 accounts. So that’s pretty cool.
And the final thing I added was a 1 time payment option.
This is something I was going back and forth on, but I realized that some people might not want to pay $5/month for automatic maintenance, but would pay a 1-time $5 fee to unfollow accounts in bulk, because after that, they may not need to bulk unfollow. And I didn’t want to make them sign up for the $5 plan for a single month and then downgrade. I want to make this as user friendly as possible, because I don’t believe in trapping people into subscriptions they don’t need.
So it added a little bit of complexity to the app to handle this extra case, but I’m happy I put in the work to do so.
Now, I think the app is feature-complete enough to launch. I’m on vacation this week, so I’m not going to do a Product Hunt launch, but I’m doing a soft launch to hopefully uncover any other issues that might come up, but I’m pretty confident at this point that things work as expected. I do still have to add in the stuff that will fire on a monthly basis, but I figure I’ve got a month to add that.
Feather Updates
Did run into some issues with Feather as well. Unfortunately, one of the services I depend on is now shut down. I used Quirrel as a way to schedule things and run all sorts of processes, but they were acquired and I knew it was coming. I was hoping to have everything ready to migrate over to the Django backend, but I didn’t quite make it. I made the decision over the weekend to attempt to set up a self-hosted Quirrel instance and swap it over and let’s just say…it didn’t go well. In fact, it didn’t go at all.
This is due to a number of factors, but mostly my deployment setup. If I had Feather deployed in the same service as my Redis instance, I could have made it work, probably. But since I don’t, I wasn’t able to get Redis, Quirrel, and Feather to connect to each other. I thought about setting up another Redis instance with Flight Control and getting everything deployed through there, but that would have ended up ballooning my costs by quite a bit and the small tests of that setup I tried didn’t lead me to believe it would have been a simple task.
So Feather is a bit dead in the water right now until I can get that fixed. Goal is to have it back up by next weekend after I get back from vacation. I’m going to have to move the functionality to the shared backend and get it all connected, and then probably move the data over as well, which I’ve got a plan to do. We’ll see what goes wrong with that, but I’ll make something work.
Build In Public University Updates
Well, as I said, decided to relaunch the blog using the hosted Ghost instance and did get that set up. And I published the Build In Public Manifesto. This will be the guiding principles I follow because I believe deeply in them.
And if you prefer, I did also set up the YouTube channel and created a video of walking through the Manifesto as I was working on getting it developed.
Also, as a side note, if you check out the graphics on the blog, I’m having a ton of fun with Midjourney. AI generated art is going to keep getting bigger as time goes on. I don’t think I’ll ever use stock images again because I’m having too much fun generating my images.
Just for fun, here’s some of the ones I’ve created:
This first one is the way I imagine Build In Public University and it has become the defacto logo I use for it.
For this next one, I wanted to think about the creators of the future signing the Build In Public Manifesto and so this is what I came up with:
Finally, here is one that I created to submit for consideration to a friend’s VR Art Gallery that she’s creating:
I’ll be continuing to generate images because as I said, it’s just a lot of fun! And it makes me feel like I’ve got artistic abilities to an extent I never did before.
Until next time, keep building! And let me know how I can help!
~Leo