This was my first and only that I've done so take this for what it's worth:
I've been reluctant to join one of these (and the Halls of Fame) just because my brain isn't wired properly for ranking things. Like, if you asked me to rank the four Indiana Jones films I could manage that. Ask me to rank the 11(?) Star Wars films and my brain will crash. Give me an even more open-ended topic (Top 50 favorite movies) and I wouldn't even be able to get started on that. I gave this a try anyway and while I enjoyed the "finding new artists" part, the ranking process was just as difficult as I expected. That last 16-song round was a nightmare.
So having said that, I'd be much more inclined to join a themed tournament in the future. Give me 10 jazz songs and I can come up with some sort of ranking. Give me one bluegrass song, one video game score, one jazz tune and a Miley Cyrus song and my brain doesn't know how to process that. It's beyond "apples and oranges" for me and more like "apples and lasagna".
So when my participation started to dwindle at the end it was because I'm a basket case, not because the tournament wasn't fun or well-run.
I don't know if this will help or not, but when ranking the songs, (or movies for the HoFs), try to remember that there are no right or wrong answers. It's all a matter of opinion, and your opinion is just as valid as anyone else's opinion.
Personally, when ranking lists, rather than trying to rank the whole list when I'm finished listening to all the songs, (or watching all the movies), I find it easier to just break it down and deal with each item one by one. I just start with the first two items on the list, and decide which of the two is my favorite, and then put that on top. Then I take the next item, and starting at the top of the list, I just compare it to each item on the list one at a time. By starting it at the top of the list, I am comparing it to my current favorite, so when I get to the item that I like it more than that item, It will just kind of fall into place where it belongs.
For me, doing it one at a time takes away the stress of having to figure out everything all at once.
Having said that, I know how you feel about ranking lists because I feel the same way when I have to write reviews of movies for HoFs, (or any other reason for writing a movie review). I have a problem with "the blank page". It can take me longer to write a one paragraph review than it took me to watch a 2-hour movie. I can stare at the blank page for hours, and not know what to write. I usually know if I liked a movie or not, but I can't always express in words why I liked or disliked a movie, and I can have a panic attack trying to figure it out sometimes.