Per-song album covers

[tearfulnumpty]

Description

Let's assume we have song A, and song B. Both have the same artist tags, and the same Album tags. However, their art tag is different.

Currently, Auxio's behavior seems to show that if the artist and album tags match between two given songs, there is an assumption that the two art tags for said songs are also matching. Thus, in the instance where the art is NOT identical between the two songs, as is the case for the song A and B example, an incorrect image is likely to be displayed when the song is playing. Of course, the likelihood of this changes based on the number of songs with matching album and artist tags. If there's only two matching, then only one song will be displaying the wrong art (as the other would be displaying it's own). If there's 100+ matching as is the case for me with just one pair, it's a big issue.

In addition, some of my songs without art tags will end up with an art tag displayed which is from another song altogether. This is a minor annoyance to myself and I can live with this part, but I assume it's related to the rest of the above stated behavior so I wanted to mention it as well.

---

I realize that this functionality may be for performance reasons, and it may be more desirable for some users to function this way. However, I run into this problem on a regular basis, and I can't imagine that I'm the only person in this situation who would desire accuracy of tags over the speed of a rare/uncommon library reload occurrence.

So, my proposal would simply be an additional setting, similar to the existing, "Settings -> Content -> Images -> Album Covers" settings. The way I see it, either an additional option could be provided for the above stated setting, or a simple on/off checkbox could be added nearby under the, "Content" grouping of settings to not automatically assume all tracks within an album by the same artist have the same cover art.

Problem solved

It's frustrating to see the incorrect art for a track, and I run into this problem on a very regular basis as it impacts a rather large amount of my library. It makes me wish I hadn't tagged my songs by artist in the first place sometimes.

Other implementations

While multiple strategies exist (performance vs quality and accuracy), other players that I have used in the past use the tags existing within the given song, and don't make assumptions like this - including mobile players.

Benefit

This would benefit Auxio users by allowing a simple option to prioritize performance in the one time library load, versus prioritizing quality and accuracy of the track's art. Most users aren't reloading their libraries frequently enough to see massive performance impacts, and those who do will of course always have the existing setup and options - no harm done with a (presumably) relatively simple addition of a quality of life option.

Duplicates

• I have checked this feature request for duplicates.
• I have checked that this feature is not implemented in the lastest version.
• I have checked the Why Are These Features Missing? page.
• I agree to the Contribution Guidelines.

[OxygenCobalt]

Very good observation. This is largely a consequence of Android's cover art source doing this kind of dumb grouping by album, and me retaining the behavior since I usually flatten the cover art of an album.

I think the settings approach is a good idea, albeit it could also become an intrinstic behavior to the "Quality" Album Covers mode. My primary issue as it stands is figuring out how to still leveraging in-memory bitmap caching functionality, since I'll no longer be able to depend on the album to cache a single bitmap (Perhaps I'll use a SHA-256 hash?).

[tearfulnumpty]

Of course it falls back to Google's design in the end, haha. It seems that this is a common trend for Android audio players.

Unfortunately I don't think I'd be of much help to suggest technical solutions. I was hoping it would be as simple as referencing the current song's actual tags instead of utilizing caching and relying on Android functionality, but I guess that kind of defeats the purpose of, "library loading" in the first place.

If you were to hash to compare images, perhaps a more performative hashing strategy could be used? It's not like using something which is considered, "insecure" these days (eg MD5) is a real problem for this use case.

Alternatively, or in addition to the above, perhaps only hashing a small portion of the image could result in higher performance? Or better yet, if there's a way to measure file sizes of the image stored in tags, you could first check to see if the file sizes match, and only compare a hash if they have identical file sizes (which should be an edge case in the first place, I would strongly presume).

This isn't exactly my expertise, so apologies if these aren't useful suggestions. I'm trying to help where I can, though I recognize it may not be the most useful.

[OxygenCobalt]

Of course it falls back to Google's design in the end, haha. It seems that this is a common trend for Android audio players.

Yep. Android's media framework (and filesystem API writ large) is notorious for being borderline unusable. I've been progressively trying to expunge as much reliance on the bugged system code as possible (Read: #322, #128) as time goes on.

I was hoping it would be as simple as referencing the current song's actual tags instead of utilizing caching and relying on Android functionality, but I guess that kind of defeats the purpose of, "library loading" in the first place.

I mean, that's what the "Quality" album cover mode does. It's just slower and uses more memory. It's generally much faster if you leverage a link to an image rather than parsing image data out. I want start using this everywhere (with my own independent image code) with #327, but the specifics are not exactly well-defined on how this works.

As for the caching bit, Auxio uses a library that keeps loaded album covers in memory for a little while so I don't have to keep re-loading them from songs or the system. This is super nice, but it relies on a "unique id" for loaded album covers, something that would have to be switched from the album ID to something a bit more unique to prevent sorting multiple versions of the same album cover.

If you were to hash to compare images, perhaps a more performative hashing strategy could be used? It's not like using something which is considered, "insecure" these days (eg MD5) is a real problem for this use case.

SHA-256 tends to be hardware-accelerated these days, making it often as cheap as MD5 AFAIK. I'm already using it for music identification and the performance loss is largely minimal.

Alternatively, or in addition to the above, perhaps only hashing a small portion of the image could result in higher performance? Or better yet, if there's a way to measure file sizes of the image stored in tags, you could first check to see if the file sizes match, and only compare a hash if they have identical file sizes (which should be an edge case in the first place, I would strongly presume).

Maybe. I'll have to do some performance analysis on which strategy is the best. Figuring this out is actually quite important, as it will be what backs #327.

[OxygenCobalt]

Actually, no, this is blocked by #327. I realized that my alternative cache ID strategy is dependent on extracting the cover art for each song, more or less negating the point of it. The only way I can do this efficiently is if I had a "birds-eye-view" of all the cover art, which only occurs when loading music, thus requiring #327.

[OxygenCobalt]

Hey @tearfulnumpty, I wanted to ask a few things about this change.

1. Would you mind this feature being implemented with the current on-demand cover system? This would be very memory-intensive, but would get you what you want faster.
2. How would the cover art of album items work if different songs have different covers? Would it be a mosaic, like artists, or would it be a single cover derived from something (i.e the first track)

[tearfulnumpty]

Sorry for the delayed response.

1. I suppose it depends on just how memory intensive the solution would be, and what factors would influence that. I wouldn't want any changes to make the app unusable for others. Me personally, I see unused memory as wasted memory. It's not like memory is a big power factor, and as long as phones made in the past 5 ish years can still handle it I think it'd be fine. Perhaps an option to control behavior would be helpful for those who this could potentially negatively impact, though I recognize that an option here probably isn't a simple thing to implement.
2. I'm not 100% sure I understand what you're asking here, so please correct me if I'm interpreting this wrong. My solution was that if different songs have different covers, that each cover art shown by Auxio should be from the corresponding song. So, song (or file) A should have cover art A, and song B should have cover art B, etc. Nothing fancy with any mosaics made from multiple files, no ignoring file C's art C and instead using file A's art, etc.

[OxygenCobalt]

I suppose it depends on just how memory intensive the solution would be [...]

It would probably be very memory intensive, hence it would be a setting with a heap of warnings surrounding it.

I'm not 100% sure I understand what you're asking here, [..]

I'm referring to albums in this case. Individual songs will display their individual cover art, but albums would have to pick some kind of cover art from the songs it contains. I was asking if there was any preferred strategy for picking a cover art to use in this case.

@tearfulnumpty

[tearfulnumpty]

I believe my phone has 4 GiB of memory, and I have no idea how it'd handle a massive memory challenge. If it was easy for you to implement, I'd be willing to try it for the meantime. I suppose with my low-mid tier phone, it may not work out for me until the proper fix with #327 comes out.

In regard to the second point, thanks for the clarification! I don't have any massive preference of any specific strategy or algorithm for picking this, no. I suppose perhaps the first track method would make the most sense to me personally.

[OxygenCobalt]

I believe my phone has 4 GiB of memory, and I have no idea how it'd handle a massive memory challenge. If it was easy for you to implement, I'd be willing to try it for the meantime. I suppose with my low-mid tier phone, it may not work out for me until the proper fix with #327 comes out.

Okay, if that memory limit's the case, I'll not go for it then. Note that it will be a very long time before you see this (1 or 2 years), since #327 is probably the most distant architectural rework I'm considering.

In regard to the second point, thanks for the clarification! I don't have any massive preference of any specific strategy or algorithm for picking this, no. I suppose perhaps the first track method would make the most sense to me personally.

Cool, I'll do that.

Thanks for the clarification.

[tearfulnumpty]

Okay, if that memory limit's the case, I'll not go for it then. Note that it will be a very long time before you see this (1 or 2 years), since #327 is probably the most distant architectural rework I'm considering.

Dang, I really hate to hear that. I totally get it's not a simple fix, but it's very frustrating all the same. Is there any way to implement something similar to your high memory usage temporary fix solution using storage space instead of memory? Something which could be done during the, "scanning music" phase? I do have a very large microSD card installed which I throw all of my media on, and I wouldn't mind Auxio taking up some Gigs of it if that would be useful, or some other type of similar caching.

I really appreciate Auxio being around, since it seems like almost all other music players have some pretty serious drawbacks and highly questionable decision making going on. Thanks for what you do.

[OxygenCobalt]

Dang, I really hate to hear that. I totally get it's not a simple fix, but it's very frustrating all the same. Is there any way to implement something similar to your high memory usage temporary fix solution using storage space instead of memory? Something which could be done during the, "scanning music" phase? I do have a very large microSD card installed which I throw all of my media on, and I wouldn't mind Auxio taking up some Gigs of it if that would be useful, or some other type of similar caching.

What you're describing is #327 @tearfulnumpty, save the grouping by hash. What makes this hard is the logistics of collating and storing cover art itself in an efficient way. While this is a fun problem, given the somewhat minor scope of this issue, I don't want to get distracted working on this at the moment, given that playlists are still only 3/4th done right now and and the app's background work system needs to be massively reworked. I mean, getting nerd-sniped by #128 pushed playlists back by a year, and I don't want to let that happen with other in-demand functionality again.

[RustoMCSpit]

i dont know if this helps but https://github.com/MuntashirAkon/Metro doesnt have this issue

[RustoMCSpit]

another difference is that while auxio assumes all files with the same album name tag are the same metro treats each file the songs are stored in as its own environment / scope. if i have two songs under the album tag 'beer fish' but stored in different files, metro correctly places them as different albums both called 'beer fish' while auxio assumes theyre the same album

[RustoMCSpit]

i dont know if this helps but https://github.com/MuntashirAkon/Metro doesnt have this issue

this is coded in kotlin too btw

[OxygenCobalt]

i dont know if this helps but https://github.com/MuntashirAkon/Metro doesnt have this issue

Because Metro uses JAudioTagger and requestLegacyExternalStorage to work around the performance issues of reading the cover art of every single song. It also does some compression too so that it doesn't fry old phones with low memory capacity. The former is a non-starter since I don't want to lock Auxio into Android 11 forever (something required by requestLegacyExternalStorage), and the latter is a non-starter since it defeats the entire point of high-quality cover art if you're hastily compressing it like that.

another difference is that while auxio assumes all files with the same album name tag are the same metro treats each file the songs are stored in as its own environment / scope. if i have two songs under the album tag 'beer fish' but stored in different files, metro correctly places them as different albums both called 'beer fish' while auxio assumes theyre the same album

That assumes a well-formed folder structure. I do not do this since folder structures are significantly more subjective than tags, and so such heuristics break down. For example, another person that organizes by genre would suddenly have all their albums split.

this is coded in kotlin too btw

Using the same language doesn't necessarily imply that design choices are transferable. For example, Auxio's lighter feature set and complex music model differs significantly from Retro's/Metro's rich feature set and simple music model.

[RustoMCSpit]

That assumes a well-formed folder structure. I do not do this since folder structures are significantly more subjective than tags, and so such heuristics break down. For example, another person that organizes by genre would suddenly have all their albums split.

sounds like it should be a setting in auxio then ngl

[RustoMCSpit]

what is a 'music model'

[OxygenCobalt]

sounds like it should be a setting in auxio then ngl

Not really. I don't like extensively customizing exactly how Auxio interprets music libraries unless there really is no unambiguous standard that all libraries rally around (i.e multi-value separators). Here, nearly everyone facing the problem of albums being accidentally merged used MusicBrainz IDs, and basically never by folder characteristics. I would much rather implement something like this as a heuristic in the music loader itself, but since it harms users with different library configurations, I can't really do it.

what is a 'music model'

It references to how a music player interpretation of a music library and how it should be organized into albums, artists, genres.

[RustoMCSpit]

MusicBrainz IDs, and basically never by folder characteristics

so if two albums had the same name then what

[OxygenCobalt]

Updates:

So, symphony just did this:

zyrouge/symphony@b423c21

I thought this was not possible, as android indexed cover art by album instead of song. If this actually isn't the case, then suddenly this is a lot more feasible, as I don't need to rely on the high-quality cover fetcher. Memory issues still apply though.

so if two albums had the same name then what

They would have different MusicBrainz IDs and Auxio would separate them out.

[dmint789]

@OxygenCobalt I would also love this feature. For example, Everywhere At The End Of Time by The Caretaker has six different album covers, and they change as the album progresses. It's also possible that some albums have different covers for every single track, like compilation albums. Also, there's the issue with singles sharing the same random cover sometimes, as OP mentioned. What is the current status on this issue? Also, I really love your music player, it's by far the best Android music player I've used, and I've tried MANY over the years. What's the best way to support you/this project?

[TL-P]

Hey, I'm really happy to see that this is implemented!

However, I've noticed a few bugs while testing the Auxio_PerSong.zip debug apk. I'm on Android 13.

My library has a cover.png file for albums with only one cover and embedded covers on albums with multiple covers. The cover.png for albums with only one cover has worked perfectly so far (v3.4.3), although I know that this is not yet supported by the app itself (#104).

Albums with multiple (embedded) covers: each songs correctly displays its cover, but the media notification uses the same artwork for all songs on an album.

For albums with only one cover (cover.png), the behavior is quite strange: album (albums view) and song (now playing) covers are sometimes different and can both be incorrect. The app also displays the same album art for all albums. The media notification however seems to always display the correct artwork.

I'd be willing to send screen recordings or sample files is you can't reproduce this issue @OxygenCobalt.

[OxygenCobalt]

I can reproduce basically none of those behaviors @TL-P, I need screenshots and sample files. Just trim the audio if it's copyrighted. Include the cover.png files.

[bastianilso]

Thanks @OxygenCobalt for your efforts towards solving this! I also tested the Auxio_PerSong.zip build with my own music collection (located on an SD card in a Fairphone 3, running LineageOS 20, Android 13).

Results (cover.jpg / embedded art):

• ✅ / ✅ Lockscreen media player shows the right album art for the song being played in all cases. Files without embedded album art display the album art belonging to the album (e.g. the folder.jpg/cover.jpg): screenshot. Files with embedded album also correctly prioritize to display the embedded album art over external images.

• ⚠️ / ✅ Bottom sheet (folded): Incorrect album art is shown if the played file has no album art (or the album art only is available as cover.jpg). The shown album art does not belong to the album (seemingly random album art gets picked? not sure). screenshot. Correct album art is shown if the played file does have embedded album art. screenshot

• ✅ / ✅ Bottom sheet (expanded/Now Playing): Correctly shows the right album art in all cases and also if the album art only is available as cover.jpg). screenshot

• ✅ / ✅ Album view (top): with no embedded album art, the correct album art is shown at the top. screenshot.

• ⚠️ / ⚠️ Album view (song thumbnails): thumbnails in the album view's song list never seem to display album art in Auxio's album view? (not sure if by design). screenshot

• ⚠️ / ✅ Artist view (artist list thumbnails): same issue as bottom sheet, if files have no embedded album art, the wrong album art is shown: screenshot. However, for all albums containing files with embedded album art, the correct album art is shown. I confirmed this also by embedding cover.jpg into one of the affected albums in the image above, which resulted in the right displayed album art when i rescanned the music library. For multi-art albums, thumbnails display correctly.

So in conclusion with my own collection: Files with embedded album art now correctly show their art everywhere. Files without embedded album art show incorrect album art in small thumbnails like the list views or folded bottom sheet, but show correct album art (cover.jpg) in album view, lockscreen and expanded "now playing" bottom sheet.

Reproduction:

To reproduce the behavior, i have created two zip files, they contain two CC-BY songs. "Nangdo_issue.zip" contain two songs with no embedded cover art. "Nangdo_fixed.zip" contains the same two songs, but with cover art embedded. I also appended "_fixed" and "_issue" to artist and album tags. In the Auxio build, the "Nangdo_issue.zip" takes on a random album art in the artist list view, while "Nangdo_fixed" correctly displays the albums art. screnshot

Nangdo_fixed.zip
Nangdo_issue.zip
Songs by Nangdo, distributed under the CC-BY license.

[OxygenCobalt]

Wait. I think I get it. I'm not properly keying album art when there is nothing embedded in the file that I can obtain a perceptual hash from. To fix that I need to fall back to UID, which will be quite inefficient @bastianilso. However, since most people use embedded album art (At least, I hope so...) this should be okay. You'll probably face increased memory usage until I can do #104.

[OxygenCobalt]

Auxio_PerSong2.zip

Updated build with that embedded cover art fix (I think). Don't have the time to test right now, but you can try it out @bastianilso @TL-P

[bastianilso]

@OxygenCobalt Almost fixed it! All album artwork now correctly show for the respective album. But a side-effect of that fix is that albums without embedded art show a 2x2 grid of the same album artwork.
album list view screenshot
The change also affected album view. However the bottom sheet is now fixed in both folded and expanded mode.
album view + folded bottom sheet screenshot
expanded bottom sheet screenshot

Edit: and regarding inefficiency, yea fine for most use cases i think? Personally, my music library strive to have individual artwork embedded, so I wont mind. I mostly have cover.jpg as fallback for 'simpler' tag-less browsing as my music collection is partially synced between a wide range of devices.

[OxygenCobalt]

Oh, I should have seen that coming. I'm just going to assume that if I could not find a song's cover, I should fall back to the album cover.jpg URI instead, given that this only occurs when using cover.jpg or with possibly unsupported metadata formats (not likely). Will do that.

[TL-P]

Thanks @OxygenCobalt for the quick fix and @bastianilso for saving me some time :)

After testing Auxio_PerSong2.zip for a little while, I can confirm that everything is working for albums using cover.png, except for the issue that @bastianilso mentioned:

albums without embedded art show a 2x2 grid of the same album artwork

However, this issue that (I assume only) I've encountered has not been resolved:

Albums with multiple (embedded) covers: each songs correctly displays its cover, but the media notification uses the same artwork for all songs on an album.

This is a screen recording of what I mean:

screen-20240422-203858.mp4

Not sure why this is happening. Could this be OS specific? (I'm running PixelOS, Android 13.)
Again, I can send sample files if you want @OxygenCobalt.

[OxygenCobalt]

That's giving me memories of a really bad notification desync issue I had to debug awhile back. It's more likely to be a consequence of #37 rather than this. Let me check the sample files.

[TL-P]

Okay, I've replaced the audio with 5 seconds of silence and included tracks 1-3 in the folder.
I could reproduce the issue with these sample files:

Auxio Samples.zip

[OxygenCobalt]

New build that should hopefully have fixed non-embedded album art: Auxio_PerSong3.zip (Still no time to test) @bastianilso

Also: That issue is indeed device-specific @TL-P. I'm just going to have to ram my head and try random things until Media3 gets the memo. Try this: Auxio_PerSongVariant.zip

[bastianilso]

@OxygenCobalt ✅ did a quick test and it all looks good to me now. Might not be worth anything, but in artist view I'm not sure how Auxio decides whether a 2x2 grid is employed as thumbnail or a single album art piece is employed as thumbnail for different artists. But personally having consistency there doesnt really matter much to me.

[OxygenCobalt]

decides whether a 2x2 grid is employed as thumbnail or a single album art piece is employed as thumbnail for different artists. But personally having consistency there doesnt really matter much to me.

It depends on the amount of covers available. If there are more than 4 distinct covers, a mosaic is shown. Otherwise, just one is shown. I've been wanting to change this a little actually, but haven't decided on what.

[TL-P]

Try this: Auxio_PerSongVariant.zip

Unfortunately, the issue is still present. The behavior changed a bit though (the output switcher now also uses the same incorrect cover art as the media notification):

screen-20240423-135405.mp4

[OxygenCobalt]

Weird @TL-P. This implies either:

1. Media3 is bugged
2. The hacky stuff I did on top of Media3 is bugged.

Will need to debug both.

[OxygenCobalt]

Okay @TL-P, I've clobbered the notification code so it's all being managed by Media3 and doesn't have odd extensions. If this doesn't work, it's Media3's fault and I have to figure out how to reproduce it with their demo app. Otherwise, it's my fault and I have no idea why.

Auxio_PerSongVariant2.zip

[TL-P]

Well, this is interesting. Auxio_PerSongVariant2.zip behaves the same as Auxio_PerSongVariant.zip.

[TL-P]

Okay, this might help: I've tested the samples with VLC for Android and symphony and both work!

[tearfulnumpty]

Realized that there may be a far easier way to do this, assuming zyrouge/symphony@b423c21 works.

Basically, I take the (perceptual) hash only and then just store that with the song. Then I use that URI to get the song's album cover and then key it to the hash. This circumvents needing to store the image (not that it wouldn't be useful, faster load times and higher quality covers are still good), but it allows me to deliver this much faster @tearfulnumpty.

Still on track for 3.5.0, just need to pry myself away from other issues and deliver this.

So excited for this! Thank you very much for seeing this through, it looks like it was certainly no easy task, but I really appreciate it!

[OxygenCobalt]

Sorry for taking so long @TL-P, need to do school stuff. Think I figured out the issue though.

[TL-P]

Good to hear! Honestly, I'm just happy you're doing all the work in your free time @OxygenCobalt :)
I'll try to test new builds if needed.

[OxygenCobalt]

Alright @TL-P, I believe I identified the issue as a mismatch between two cover data sources that were conflicting with eachother. I fixed it at the cost of making it so that the notification can no longer respond to music settings, and I guess I won't be able to re-add that until #327.

Auxio_Canary.zip

[TL-P]

Can confirm the cover issue is fixed now!

Side note: I noticed the updated UI in this build, looks nice. However, round mode seems to be enabled even when the setting is turned off?

[OxygenCobalt]

Yeah @TL-P I'm still working on the new UI, I just accidentally worked on this patch in that branch. It's still under construction (I haven't planned out how the UI will change when round mode is off)

[TL-P]

Just one question: Can you explain what you mean by "the notification can no longer respond to music settings"?

Thanks!

[OxygenCobalt]

@TL-P basically, if you change Album covers or turn on 1:1 album covers, the notification cannot respond to it. It'll always be non-1:1 and possibly not be high quality.

[OxygenCobalt]

Looks like due to #802 I'm going to have to modify the image keyer a little. A perceptual hash will be so horribly inefficient that it will make libraries with large cover arts unusable. I think I'm just going to randomly pick a string of bytes in the middle of a cover data stream. This means if you use different images for your cover art they will end up being duplicated in memory, but until #327 (which itself needs to be retooled aggressively) I can't risk tanking music loading performance even more.

[dmint789]

@OxygenCobalt I'm a bit confused, so is this done or no? Because it says closed, but you're also saying you need to do something else first.

[OxygenCobalt]

This is shipped in 3.5.0 @dmint789 but in a less-than-ideal state.

More or less:

• If you have album covers of inconsistent format in your library, they will be duplicated in memory and it'll be inefficient
• If you somehow have two distinct album covers that have the exact same byte sequence halfway through the file, then they will be coupled in memory and the covers wont display correctly

Hence, I need to make the system more robust, but these edge cases are rare enough for me to push it back.

[dmint789]

Thank you, I tried it out and it works great!