I posted a video on YouTube today with where we’re going in Micro.blog to use AI in a limited way to help with search and accessibility.
@weirdwriter Opt out feels right to me. It's useful and most people will benefit from it, but we don't force it on people if they don't want it.
@manton Love it. Re:PNG, always wanted to ask screenshots need alt text or if screen readers can read them as is. Also, to confirm, the alt text and keywords will be eventually added to older uploads, right?
@pratik Yes, I want to do more testing with screenshots before enabling it for PNGs, but it'll happen later. I need to see how everything performs with a variety of real-world blogs first, and how processing older images affects costs.
@pratik That's correct. Right now the alt text that you use in a blog post is only stored with the post itself. Things will get merged together in the future, so that blog posts and uploads are better connected.
@pratik @manton Correct, I can’t save anything to Micro.blog to reuse later with Shortcuts (see my text file workaround for Markdown Memes). But with the new availability of the generated alt returned by M.b’s uploading API, I have some ideas on updating my shortcuts to use those primarily and only falling back to the OpenAI API if necessary (i.e. Non-premium users or for PNGs). 😁 Exciting stuff!
@jarrod Cool, lemme know if you have any questions or if anything is missing! Because these are generated in the background, the main limitation right now is that I can't immediately return the text at the time the image is uploaded.
@manton will these features always be only for Premium subscriptions, and will it one day trickle down to Basic users?
@manton Ahh, yeah, that might be a showstopper. I suppose that field would be more for third-party apps that load and show all your uploads?
@jedda For the foreseeable future they will be limited to Premium subscribers, mostly because we're taking on new hosting costs ourselves. Hopefully we can make more of this available to the standard plan, but not this year.
@jarrod Yes, although you could also poll the uploads list a few seconds later to get the new text. Not really ideal, though.
@manton I’m sorry if you’ve answered this but I didn’t hear it in the video. If I upload a photo and post it, does it the alt text get generated in the background and update the post?
@gregmorris Nope, it does not update the actual post automatically yet. For now it’s a manual process to copy the text and use it.
@manton that’s not the end of the world I suppose, it not ideal. Bit too much of a faff when posting from mobile but it’s a nice improvement none the less.
@gregmorris Stay tuned, we'll expand this. Need to let the dust settle a little bit first with the new features added today.
@manton sure would be nice if you could post your video on your blog 😁
This is seriously a cool feature though. I struggle with how to deacibe my photos sometimes, and this gives a quickstart. Hope this makes it to all the interfaces and we get warnings about failed uploads, too.
@manton love it. Great. Would really like to have the alt text automatically there on a photo I attach to a post. Also hoping the search works on simple filename too ?
@manton I see it is hard at work! “Summarizing JPEG photos... 1315/8429”. 😁 The robots are sweating!
@manton Just wanted to point out that you announced an accessibility feature with a video that visually disabled users can’t see, Deaf/hard of hearing users can’t hear, and where the only textual alternative is the janky transcript of YouTube’s automatic captions with not even a cursory once-over edit pass to correct egregious errors, add punctuation, or generally make it legible. There’s more to supporting accessibility than adding an AI feature that might be able to describe an image, but won’t be able to take into account any of the context of why a particular image is used or if there are particular aspects of the image that are of specific import.
@djwudi Thanks for the feedback. I’ll go back and improve the transcript today. I don’t love using YouTube and want to bring these videos into my own blog later.
@manton No, it hasn't made that much progress actually. "Summarizing JPEG photos... 1511/8430". Processed 196 photos since that earlier reply.
@manton FYI as I tried the AI podcast transcript option (auphonic) but turned it off again for environmental reasons but the Use AI check box was checked on for me. I’ve turned it off now but just letting you the off for current users flag maybe needs tweaking ?
@adamprocter I think that sounds correct, unless I'm misunderstanding… If you had ever used the transcripts, we enabled the "use AI" checkbox automatically. But it won't come back on if you toggle it off.
@manton from what I can tell I turned it off soon enough for it to have only processed 16 photos of my 2000+
@manton yes your correct. I turned it off search says it processed 16 photos or 2000+ and asks me to turn AI back on. So I think I have stopped it from doing the work
@adamprocter Thanks, I don't think I perfectly handle turning it off in the middle of processing. I'll improve that.
@manton Currently "Summarizing JPEG photos... 1427/8430". It is odd that when I have that page open the number of images summarized increments about every 4 seconds. But clearly it isn't going that fast otherwise it would be much further along. And I can’t imagine it would matter if I have the page loaded or not?
@manton image processing seems to have stalled out at 17/1274 for the past day or so. Is there a way to reboot?
@mroutley Looks like there are some errors while processing the photos. Hope to have a work-around soon, and it'll automatically kick the process off again after that.
@manton Thought I would share that for me the AI generation is still just sort of stuck in my archive. Currently "Summarizing JPEG photos... 1514/8438". When I have the page open it moves.
@jthingelstad I think this is back on track for your account. Still going to take a while... I'm working on improving how fast this is.
@jthingelstad Haha! It's like the opposite of a 'watched pot' 🙃 Reminds me of my brother, growing up, who would start doing his homework from school as soon as my dad came home.
@manton I suspect you are right, it is up to 1635 now. My quick math says it will be done in a little under 12 days. No worries from me. It is a one time process. 😲