manton
manton

Sometimes I get carried away using Redis. Made another change a couple days ago that shaved over 12 GB off our memory usage. Probably more we can do.

|
Embed
Progress spinner
condie@hachyderm.io
condie@hachyderm.io

@manton do you have an explainer of your micro.blog tech stack in blog or podcast form? I’m working on a project that will have similar needs and I’m curious what approach you took

|
Embed
Progress spinner
Miraz
Miraz

@manton Must say I've noticed recently that the Timeline is speedy and publishing to my blog is also speedy. You're doing something right. 😁 Thank you.

|
Embed
Progress spinner
manton
manton

@Miraz Great to hear!

|
Embed
Progress spinner
manton
manton

@condie Not really in one place… It's Ruby Sinatra for the web platform, Hugo for blog publishing, MySQL for most storage, also S3 for photos, Redis for storing lists of posts in the timeline. I posted a server diagram a few months ago too.

|
Embed
Progress spinner
pratik
pratik

@Miraz @manton Yup. Publishing was taking 15-20 seconds. Now < 5.

|
Embed
Progress spinner
manton
manton

@pratik 👍

|
Embed
Progress spinner
starrwulfe
starrwulfe

@manton I'm grateful for your memory management work. Mine is terrible apparently.😅

|
Embed
Progress spinner
SimonWoods
SimonWoods

@manton @condie There's also How does Micro.blog even work?

|
Embed
Progress spinner
manton
manton

@SimonWoods @condie Oh, thanks! I forgot about that post.

|
Embed
Progress spinner
jgarber@mastodon.cc
jgarber@mastodon.cc

@manton Super curious to know what the change was that resulted in such a dramatic improvement!

|
Embed
Progress spinner
manton
manton

@jgarber I've been making some smaller changes, but this one was kind of dumb... I was storing some logging data in Redis and it got completely out of control without me realizing it. Moved it to MySQL.

|
Embed
Progress spinner
jgarber@mastodon.cc
jgarber@mastodon.cc

@manton Oh, interesting. Glad you found an easy solution!

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton I wonder if one of the changes/fixes you've been working on fixed the Cloudflare/Imunify360 issues I've been having with my blog feeds. I just noticed that micro.blog can "see" my feeds again, although I think I need to manually reload them.

|
Embed
Progress spinner
manton
manton

@kimonostereo That’s great it’s working! I don’t think I’ve made any related changes but let’s hope it keeps working.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton Thanks! Me too! Will there be another release of the micro.blog iOS app soon? Hoping an update will fix whatever is stopping xml-rpc from connecting with my blog. For whatever reason, it still works on macOS and iPad.

|
Embed
Progress spinner
manton
manton

@kimonostereo Yes, updated iOS app coming this week. I need to revisit the XML-RPC problem, though, I don't think it's fixed yet.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton Awesome! Please let me know if there is anything I can do to help!

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton downloaded the TestFlight beta, and yep! Still has the same issue previously.

|
Embed
Progress spinner
manton
manton

@kimonostereo Thanks, I'm going to fire up another Movable Type server to test with.

|
Embed
Progress spinner
In reply to
starrwulfe
starrwulfe

@manton I’m also interested in this 🔖

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton I noticed that there is a link to view logs in my account timeline RSS feeds. I'm trying to figure out why this has been working sometimes and then stops. I noticed last night there was a period of time where m.b. downloaded my feed with no errors between 2024-03-02 18:35:00 and 2024-03-02 21:03:55, then I started seeing Feed: Error parsing RSS — ohmypizza.com/micro/atom.xml and SimpleRSSError, Poorly formatted feed again. Currently the feeds page states both my feeds are Error: invalid RSS feed. Are you able to see if the feeds are actually downloaded?

|
Embed
Progress spinner
manton
manton

@kimonostereo Interesting that the error changed. I'll test this again, but I'm also going to add a special error message when the response is HTML. That should help us troubleshoot.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton Thank you for helping with this. I appreciate it! RSS feeds are currently in Error: invalid RSS feed mode. Will be interesting to see if the error logs show anything interesting if/when that changes.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton I just checked and the feeds are working again. Can you see if there is anything that you can see on your end?

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton morning update: feeds are still working and the feed log shows no errors.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton update: sometime around 10:13AM HST the feeds have stopped working.

|
Embed
Progress spinner
manton
manton

@kimonostereo Thanks for the updates. Still mysterious why this works and then doesn’t. I haven’t rolled out the better logging yet, though.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton thanks! I would love to know what’s happening too. It seems just random!

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton morning update: feeds working again!

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton Looks like feeds stopped working again at around 2024-03-10 19:38:57 according to the logs. It was fine for 3 days this time!

|
Embed
Progress spinner
manton
manton

@kimonostereo I wonder if the timing is because Cloudflare's security mechanism is related to how often it's getting requests from Micro.blog. If so, maybe we could work around it by adjusting checking feeds when there is an error.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton I'm guessing there aren't many others with this issue? Do you think this is still related to Imunify360 or just Cloudflare. I was thinking about moving (again) to another webhost if it was Imunify360…

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton Hi Manton! Would you happen to have any update regarding fixes for my issues posting from Movable Type on the iOS apps?

|
Embed
Progress spinner
manton
manton

@kimonostereo Not yet, I have a couple issues still to resolve with Movable Type and Micropub support too. Sorry for the delay.

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton I started noticing a new error in the feed logs today: Feed: Unknown error 403 Forbidden, Net::HTTPForbidden

|
Embed
Progress spinner
kimonostereo
kimonostereo

@manton I'm testing out a different web host in an attempt to "fix" my issues with micro.blog not being able to get to my RSS feed. I added a test feed but noticed that it sometimes shows: "Error: invalid RSS feed" and other times it seems fine.

|
Embed
Progress spinner