Join the hunt!
(or log in)
  • Home
  • Puzzles
  • Info
  • Leaderboard
  • Stats
  • Wrap-Up
    •

    Wrap-Up (spoilers!)

    I am very honored and privileged to have had the opportunity to run my first hunt with such a formidable audience. I did not expect such a warm reception! I am grateful to everyone who participated and left feedback on the puzzles and the website!

    I want to thank everyone who worked on the hunt with me and made this all possible:

    The overall resonance to the hunt was much more positive than I had anticipated, which is very nice. On this page, I am going to try to summarize some of my experience and answer some questions we received from participants.

    Random statistics

    * We assume solving a puzzle after solving its associated meta counts as a “backsolve”. This metric isn’t perfect.

    Technical side

    We knew going in that automated e-mails were going to be a problem, as I already struggled to find an online service that would allow us to send notification e-mails to 1000 addresses when the hunt starts, when it ends, and when there’s errata. As a result, I resigned to not having mass e-mail notifications at all and was going to send out notifications only for hint responses.

    Unfortunately, I couldn’t get this to work either. I decided on using GMX for this simply because I’d used it before and knew that it supports SMTP, but early into the hunt, they locked down the account and required me to change the password, and this kept happening over and over, so we gave up. I asked their support service about it, and 5 days later I got a response saying that this is an automated anti-abuse measure and cannot be turned off.

    For the next hunt, I plan to allow users to specify a Discord webhook address so they can get hint and errata notifications within their own Discord servers.

    I overestimated solvers’ internet connection speeds when deciding to embed images directly into the puzzle pages as data URIs. I thought my internet was below average and for me even the chunkiest puzzle page (Postcards, 4 MB) only takes 2 seconds to load. However, several people complained about this, especially because you needed to load all of the images in order to see whether a hint response had come in. In the next hunt, we will put the hints on a separate page.

    I was also surprised at the volume of complaints about a supposed lack of clipboard copyability. I wish to go on the defensive about this one, for several varied reasons:

    Feedback

    Only one respondent in the feedback form said this was their first hunt, so if we scale this up to the number of participating teams, there would likely be about 6 newcomers to the world of hunting. Welcome!

    All of the images/drawings/diagrams/works of art we received for Signal Link, Face to Face and Postcards are in a gallery on their respective solution pages.

    On the questions “How do you feel about this hunt’s length and difficulty?”, the overwhelming majority (86.7% and 71.1%, respectively) responded “just right”. A minority of comments we received highlighted the somewhat wide range of difficulties presented: the hardest were rather brutal in comparison to the easiest puzzles. As one respondent put it, this “produces some whiplash”. This did show up in test solving but we didn’t have enough experience making puzzle hunts to know that it was an unusually wide range for a hunt this size.

    To the questions “Which puzzle(s) in this hunt did you like the most?”, Board State is a clear winner, followed by Postcards and Objectionable Ranking. This is quite different from the average ratings given on the website (see Stats), where the clear winners are Totally Ordinary Crossword and Rectangle Mangle. This likely shows a statistical bias: the full feedback form was probably only filled in by completionists, who tend to be more experienced puzzlers, while the star-based rating was likely given by more newcomers.

    However, the biggest surprise to me here is that the first two puzzles written for the hunt — The Nuke and Board State — were generally rated among the worst and best puzzles in the hunt, respectively.

    Another surprise is that the subjective difficulty rating given in the feedback form is wildly off from the statistics on number of guesses and hint requests. People only agree on Face to Face. But according to the statistical metrics, The Nuke is the hardest puzzle, but in the feedback form it’s only in 8th place! The situation is reversed with Circles (feedback rated it difficult, but statistics show much fewer guesses and hint requests).

    The puzzle that I had the most fun constructing, Signal Link, was also the most backsolved. Sadface!

    When asked about the meta structure, respondents tended to describe it as “a weird choice” and “a strange concept”. Only one respondent called it “clever and innovative”. It’s really hard for me to tell which novelties will be received as off-putting and which ones are seen as an interesting surprise. Most hunts’ meta structures form a balanced tree (each level is either all metas or all leaf puzzles) so I decided to shake it up and have a somewhat lopsided tree where two metas are mixed with leaf puzzles on the same level (which also played well into the mechanics of The Nuke). What I didn’t consider — and I’m grateful for one respondent to point this out — is that “some answers are effectively worth more than others” because one of the answers to an 8-feeder meta “effectively only gave 1/8 of an answer for the final meta. [...] All answers should be treated equally!” That’s an interesting way of looking at it that I hadn’t considered.

    Lots of disappointment was raised about the lack of a full story and the lack of an epilog when solving the final puzzle, resulting in a bit of an underwhelming or anticlimactic ending for some. I am not a writer and I don’t come up with stories, and when I solve puzzles I tend to find story/theming more distracting than engaging. The little story that we did have — there’s a city and 13 locations in the city have bombs planted in them — already caused frustration: solvers complained that the location information was mostly unused, even when they recognized that it did play into the final meta. This indicates to me that any amount of story I add on top of the puzzles only serves to add red herrings. So... this is not going to change unless somebody volunteers to write a story for the next hunt and also volunteers to be the scapegoat when solvers complain about the red herrings this produces.

    The limit of 5 guesses per 6 hours was too tight. We will make this more lenient in future hunts. Despite, I hope that at least a couple teams experienced a good puzzle this way that they would otherwise have backsolved.

    Controversies

    I don’t want to draw too much attention to these, but they occurred in public, so you deserve to know. There were two unfortunate implications that surfaced during the hunt:

    Assorted comments

    Assorted comments from assorted solvers!

    Apologies to the considerable number of teams who went through Face to Face smashing every single statue, hoping for something special to happen!

    Questions