[DaveBlake]

I've read the documentation on music tagging, scraping and massaging them into the database, but I'm still confused.

That is understandable, but I promise you can make a nice music library without too much pain.

The wiki, especially section 3.2 deals with the file structure of albums, both artist albums and compilation albums....

That is a reasonable place to start, but unfortuneately wiki has not yet been updated yet for v18 which supports more flexibilty in locating local art, which means that you have more options on how you arrange things. But generally a structure <album artist>/<album>/<music files> makes sense.

 

However, I also have a vast number of compilation albums that do have an album artist. For example, "A State of Trance 2018" contains 42 works by different artists, but Armin van Buuren is listed as the album artist because he's the bloke who mixed all these separate tracks into a single album.

People think of compliations in different ways and want them handled in different ways. You can tell Kodi that an album is a compilation in one of 3 ways
a) Tag all the music files with a COMPILATION tag (a non-standard itines thing that has caught on)
b) Tag music files with common album name but different artists (and no album artist). Kodi then gives it the album artist "Various Artists" (or equivalent in your language)
c) Tag the music files with album artist "Various Artists"

Approach a) means that you can have anthologies e.g. "Greatest Hits" (all songs by same album (and song) artist but culled from different releases), or the kind of album you give as an example as a compilation if you want to do so.

What Kodi does special with a compilation is list them under a separate node - Compilations - and not under the Albums node. But they will be listed under the albums by an album artist if you navigate artists > albums

 

Then there are the compilation albums that all belong to the same series. There are two dozen albums in the "Supperclub" series alone which are classified as being from "Various Artists" but are entirely different in character from a "Various Artists" collection such as "Best Ever 80s Hits" or "We are the Blules 1995". Yet, from what I understand, Kodi simply treats them all alike.

Yes, support for disc sets like that is not been developed (yet). Your best approach (for accessibility) is to split the series into separate albums. Some people I know treat the series name like an album artist (rather than "various artists"). It all depends how much of this you have and how you like to access it.

 

I would rather treat series of similar compilation albums by various artists and no listed album artist (e.g. the Supperclub or "Café del Mar" series) separately from the "mixed bag" compilation albums that are not part of their own separate group or series. From a standpoint of folder structure and file organization this is simple, but how does Kodi treat these albums when adding them to the database? I have read the Wiki, including section 7.1 which deals with compilations and multiple artists, but I'm still confused on how to treat these three different types of compilation albums.

File organisation and folder naming does not matter to how Kodi builds the library - songs, albums and artists are derrived from the embedded tags. Scanning would cope with one big flat structureless bucket, it is humans that would struggle with that! However if you want Kodi to be able to pick up local art work, or specify additional album and artist data via NFO files, then folder structure starts to matter a bit more.

Before v18 that meant a strict artist/album layout, but v18 is liberated from that for artists. The artist nfos and local art can be off in a separate structure from the music.

So how do you want to organise the music files for the different kinds of compilation you have? None of them have to be in a folder of "compilations", just tag them as you want them arranged in the library. I think you have read the old wiki and think you must do something special, you don't need to.
 

Also, Musicbrainz is inconsistent when it comes to album artist tags. Some albums by Armin van Buuren have an album artist tag listing him as the album artist, but from other albums by Armin van Buuren (which should be treated identical) the Album Artist tag
is missing.

How users enter data does vary despite attempts to enforce guidelines and produce consistency. One approach would be to submit edits to Musicbrainz to make it consistent, but that may take too much of your life! I doubt that the album artist tag is missing, I'm pretty sure that all releases have artist credits (example if I'm wrong), but that might be "Various Artists". MB guidelines don't allow for having a series name as the album artist, so it may be that MB tagging is not suited to your use. The benefits of it generally are better scraping of additional artist and album info and art, but there can be reasons to sacrifice that.
 

Section 7 of the Wiki warns me not to override this and change the album artist myself, because if I do that I'll apparently hose my database, a tree will fall on my house and I might just possibly trigger the Apocalypse.  Does that mean that I can't correct MB's inconsistencies and am forced to introduce them into my database?

Ho, yes end of the world.
It is written like that because in the early days when Kodi first started processing mbid tags some users got in a great mess and became very upset about it. Kodi assumes that these were added by software hence are consistent and complete. So if you want to change the artist or album artist tag you also have to edit the matching artist mbid tag and album artist mbid tag. Or if you decide it is too much faff then remove all mbid tags not just some of them. If Kodi finds a release mbid it will expect the artist mbid tags to be present too etc.

If it is about genuine errors at Musicbrainz then help the commity out by editing the entries. If it is about Series name as album artist then maybe remove mbid tags or edit them intelligently.

My advice is that you experiment and find out what suits you. I'm happy to explain aspects in more detail if you get unexpected behaviour. I know there are users into big sets of club music, and maybe they will share their approaches.