manton
manton

I posted a video on YouTube today with where we’re going in Micro.blog to use AI in a limited way to help with search and accessibility.

|
Embed
Progress spinner
manton
manton

@weirdwriter Opt out feels right to me. It's useful and most people will benefit from it, but we don't force it on people if they don't want it.

|
Embed
Progress spinner
pratik
pratik

@manton Love it. Re:PNG, always wanted to ask screenshots need alt text or if screen readers can read them as is. Also, to confirm, the alt text and keywords will be eventually added to older uploads, right?

|
Embed
Progress spinner
manton
manton

@pratik Yes, I want to do more testing with screenshots before enabling it for PNGs, but it'll happen later. I need to see how everything performs with a variety of real-world blogs first, and how processing older images affects costs.

|
Embed
Progress spinner
pratik
pratik

@manton Fair. Thanks for the clarification. I currently use @jarrod’s shortcuts for all images I upload to Micro.blog. But I doubt that alt text is saved at Micro.blog’s end and is only part of my posts.

|
Embed
Progress spinner
manton
manton

@pratik That's correct. Right now the alt text that you use in a blog post is only stored with the post itself. Things will get merged together in the future, so that blog posts and uploads are better connected.

|
Embed
Progress spinner
jarrod
jarrod

@pratik @manton Correct, I can’t save anything to Micro.blog to reuse later with Shortcuts (see my text file workaround for Markdown Memes). But with the new availability of the generated alt returned by M.b’s uploading API, I have some ideas on updating my shortcuts to use those primarily and only falling back to the OpenAI API if necessary (i.e. Non-premium users or for PNGs). 😁 Exciting stuff!

|
Embed
Progress spinner
In reply to
manton
manton

@jarrod Cool, lemme know if you have any questions or if anything is missing! Because these are generated in the background, the main limitation right now is that I can't immediately return the text at the time the image is uploaded.

|
Embed
Progress spinner
Eggfreckles@mastodon.mit.edu
Eggfreckles@mastodon.mit.edu

@manton will Micro.blog be able to count my almonds?

|
Embed
Progress spinner
manton
manton

@Eggfreckles I'm going to stay very far away from anything remotely food or health related. 🙂

|
Embed
Progress spinner
jedda
jedda

@manton will these features always be only for Premium subscriptions, and will it one day trickle down to Basic users?

|
Embed
Progress spinner
jarrod
jarrod

@manton Ahh, yeah, that might be a showstopper. I suppose that field would be more for third-party apps that load and show all your uploads?

|
Embed
Progress spinner
manton
manton

@jedda For the foreseeable future they will be limited to Premium subscribers, mostly because we're taking on new hosting costs ourselves. Hopefully we can make more of this available to the standard plan, but not this year.

|
Embed
Progress spinner
manton
manton

@jarrod Yes, although you could also poll the uploads list a few seconds later to get the new text. Not really ideal, though.

|
Embed
Progress spinner
gregmorris
gregmorris

@manton I’m sorry if you’ve answered this but I didn’t hear it in the video. If I upload a photo and post it, does it the alt text get generated in the background and update the post?

|
Embed
Progress spinner
manton
manton

@gregmorris Nope, it does not update the actual post automatically yet. For now it’s a manual process to copy the text and use it.

|
Embed
Progress spinner
jedda
jedda

@manton that makes sense. thanks manton!

|
Embed
Progress spinner
gregmorris
gregmorris

@manton that’s not the end of the world I suppose, it not ideal. Bit too much of a faff when posting from mobile but it’s a nice improvement none the less.

|
Embed
Progress spinner
manton
manton

@gregmorris Stay tuned, we'll expand this. Need to let the dust settle a little bit first with the new features added today.

|
Embed
Progress spinner
toddgrotenhuis
toddgrotenhuis

@manton sure would be nice if you could post your video on your blog 😁

This is seriously a cool feature though. I struggle with how to deacibe my photos sometimes, and this gives a quickstart. Hope this makes it to all the interfaces and we get warnings about failed uploads, too.

|
Embed
Progress spinner
jthingelstad
jthingelstad

@manton love it. Great. Would really like to have the alt text automatically there on a photo I attach to a post. Also hoping the search works on simple filename too ?

|
Embed
Progress spinner
manton
manton

@jthingelstad I want to add filename and date search, but it’s not there yet.

|
Embed
Progress spinner
manton
manton

@toddgrotenhuis Thanks! Yeah, it’ll be more integrated into the posting flow later.

|
Embed
Progress spinner
jthingelstad
jthingelstad

@manton I see it is hard at work! “Summarizing JPEG photos... 1315/8429”. 😁 The robots are sweating!

|
Embed
Progress spinner
djwudi
djwudi

@manton Just wanted to point out that you announced an accessibility feature with a video that visually disabled users can’t see, Deaf/hard of hearing users can’t hear, and where the only textual alternative is the janky transcript of YouTube’s automatic captions with not even a cursory once-over edit pass to correct egregious errors, add punctuation, or generally make it legible. There’s more to supporting accessibility than adding an AI feature that might be able to describe an image, but won’t be able to take into account any of the context of why a particular image is used or if there are particular aspects of the image that are of specific import.

|
Embed
Progress spinner
manton
manton

@djwudi Thanks for the feedback. I’ll go back and improve the transcript today. I don’t love using YouTube and want to bring these videos into my own blog later.

|
Embed
Progress spinner
manton
manton

@jthingelstad Just curious, did it finish churning through all 8000 photos?

|
Embed
Progress spinner
jthingelstad
jthingelstad

@manton No, it hasn't made that much progress actually. "Summarizing JPEG photos... 1511/8430". Processed 196 photos since that earlier reply.

|
Embed
Progress spinner
manton
manton

@jthingelstad Thanks, I'm looking.

|
Embed
Progress spinner
adamprocter
adamprocter

@manton FYI as I tried the AI podcast transcript option (auphonic) but turned it off again for environmental reasons but the Use AI check box was checked on for me. I’ve turned it off now but just letting you the off for current users flag maybe needs tweaking ?

|
Embed
Progress spinner
manton
manton

@adamprocter I think that sounds correct, unless I'm misunderstanding… If you had ever used the transcripts, we enabled the "use AI" checkbox automatically. But it won't come back on if you toggle it off.

|
Embed
Progress spinner
adamprocter
adamprocter

@manton from what I can tell I turned it off soon enough for it to have only processed 16 photos of my 2000+

|
Embed
Progress spinner
adamprocter
adamprocter

@manton yes your correct. I turned it off search says it processed 16 photos or 2000+ and asks me to turn AI back on. So I think I have stopped it from doing the work

|
Embed
Progress spinner
manton
manton

@adamprocter Thanks, I don't think I perfectly handle turning it off in the middle of processing. I'll improve that.

|
Embed
Progress spinner
jthingelstad
jthingelstad

@manton Currently "Summarizing JPEG photos... 1427/8430". It is odd that when I have that page open the number of images summarized increments about every 4 seconds. But clearly it isn't going that fast otherwise it would be much further along. And I can’t imagine it would matter if I have the page loaded or not?

|
Embed
Progress spinner
jthingelstad
jthingelstad

@manton thanks. As you know filename is much desired.

|
Embed
Progress spinner
mroutley
mroutley

@manton image processing seems to have stalled out at 17/1274 for the past day or so. Is there a way to reboot?

|
Embed
Progress spinner
manton
manton

@mroutley Looks like there are some errors while processing the photos. Hope to have a work-around soon, and it'll automatically kick the process off again after that.

|
Embed
Progress spinner
manton
manton

@mroutley Should be resuming now. Thanks!

|
Embed
Progress spinner
mroutley
mroutley

@manton yes, making progress!

|
Embed
Progress spinner
jthingelstad
jthingelstad

@manton Thought I would share that for me the AI generation is still just sort of stuck in my archive. Currently "Summarizing JPEG photos... 1514/8438". When I have the page open it moves.

|
Embed
Progress spinner
manton
manton

@jthingelstad I think this is back on track for your account. Still going to take a while... I'm working on improving how fast this is.

|
Embed
Progress spinner
pratik
pratik

@jthingelstad Haha! It's like the opposite of a 'watched pot' 🙃 Reminds me of my brother, growing up, who would start doing his homework from school as soon as my dad came home.

|
Embed
Progress spinner
jthingelstad
jthingelstad

@manton I suspect you are right, it is up to 1635 now. My quick math says it will be done in a little under 12 days. No worries from me. It is a one time process. 😲

|
Embed
Progress spinner