DrawAFish.com Postmortem

(aldenhallak.com)

388 points | by hallak 4 days ago ago

102 comments

  • ofjcihen 4 days ago ago

    I was one of the “lucky” few to witness the school of slur-fish.

    Being in security I laughed because of how egregious it was but also because I knew someone on HN with some actual time on their hands to help properly would be along soon.

    I also appreciate this post mortem. Vibe-coded anything in prod is a lot of my work load in IR these days but it was nice to see such a low stakes project properly documented.

    • bombcar 4 days ago ago

      People will be quick to jump on the "it was vibe coding's fault" but at least two of the issues are pretty common even in designed systems without AI - leaving in a "test admin" access and verifying tokens but not cross-checking them.

      • JohnMakin 4 days ago ago

        This is pretty reductive of the actual problem people typically complain about with vibe coding - It produces very workable prototypes fairly quickly and without a lot of hassle. Great! The problem is, and this is a great example (of many) where someone mistook the working prototype with a system that was ready for production. The JWT thing in particular is not really a mistake many people who work on that kind of thing would make.

        People need more understanding of the risks of vibe coding and YOLOing to prod with these tools. They are powerful, but like all powerful tools, can be wielded irresponsibly.

        • serf 3 days ago ago

          it's just incompleteness -- a human issue.

          most in-use LLMs prompted with a simple "You're in charge of infrastructure security, let's review possible problem points" would have uncovered this.

          I wouldn't fault a compiler for erring when someone left out a period; i'd tell the person to start including it -- but for some reason the expectation for LLMs is hands-off work ; I guess we're just in that phase of the hype at the moment.

          • bccdee 3 days ago ago

            > I wouldn't fault a compiler for erring when someone left out a period

            I'd fault it if it silently injected multiple serious vulnerabilities.

          • akdor1154 3 days ago ago

            > for some reason the expectation for LLMs is hands-off work

            The expectation is the same as the expectation for self driving: users expect it to be fully hands off, even when they are explicitly told they need to keep their hands on the wheel.

            This is because it's tricky, tedious, and unejoyable to thouroughly vet the actions of a machine in realtime.

            • floydnoel 2 days ago ago

              very interesting- i actually enjoy monitoring claude code and telling it when it is going the wrong way on something. i also don’t mind monitoring the car doing its lane keeping, perhaps it is an autism trait?

              • bombcar 15 hours ago ago

                Sorry to be the one to tell you, but you might be a born manager ;)

      • autoexec 4 days ago ago

        I think it's pretty reasonable to expect AI to produce systems with issues "pretty common even in designed systems without AI" because that's what AI was trained on.

        • JohnMakin 3 days ago ago

          But that isn’t the expectation or what is being marketed

          • edoceo 3 days ago ago

            I expect these AI and LLM to be, basically, a middle of the bell-curve type producer of code. Just like their other output. Not terrible, not exceptional, just what a Mid could do - only faster.

            Not sure what's being marketed, but I expect mediocre.

          • brookst 3 days ago ago

            Being marketed by who? Be specific.

            • ofjcihen 3 days ago ago

              “Be specific.”

              Am I the only one that feels like it’s really condescending when people say this on the internet?

              It sounds like something you would see on a community college writing assignment

              • ungreased0675 3 days ago ago

                It does sound condescending. I think the sentiment is important though. Asking someone to be specific can help them think clearly. What’s a nicer way to do that?

                • JoshuaDavid 3 days ago ago

                  I think "Who, specifically, claims that [...]?" comes off as less condescending than "Who claims that [...]? Be specific." just by virtue of the latter using imperative language, which triggers a reflexive "you're not the boss of me" reaction.

                  • john01dav 3 days ago ago

                    The message is clear in both cases. It's easier to put aside these irrational reflexive reactions and think about whatever worth can be derived from the message than it is to carefully manage the emotions of varied readers whom you don't know. This is different from bring overtly inflammatory, although the lines for this are subjective.

                    • JoshuaDavid 3 days ago ago

                      Ultimately it's probably not a productive use of time to be commenting here at all from a strict EV perspective. Meaning that if you're posting here, you're probably getting something else out of it. The value of that "something else" determines how you should approach the problem of managing the gut reactions of your readers.

                      If someone asks for a better way to word something to reduce reader hostility to their point, I assume that they will be better off for knowing the answer to that question, and can decide for themselves whether they want to change their writing style or not - and, whether they do or do not, the effects of their writing will be more intentional.

                    • comfysocks 3 days ago ago

                      In the two cases, the meaning of the message may be the same, but the tone of the message is different. One tone invites further engagement, the other invites disengagement.

            • degamad 3 days ago ago
            • bombcar 3 days ago ago

              By Brian Sheltzer, 302 Main St, Chicago, Illinois.

            • throwanem 3 days ago ago

              'By whom.'

      • siva7 3 days ago ago

        I thought checking a token against the cert is actually called verifying or is noawadys verifying just if it looks like a token it maybe a valid token?

    • stavros 3 days ago ago

      Not even a screenshot, I really wanted to see the swastikarp.

      • throwanem 3 days ago ago

        I actually did see one, while the site was #1 and well before the overnight excitement. (Good grief, even at this late age I have something to do with my Friday evenings...)

        I don't really know what you want me to tell you about it. The swastika per se as I recall had to be drawn backwards, because there is no meaningful overlap between its outline and that of a fish, so unlike the penis case this is very easy for the classifier. It wasn't clever and it wasn't funny. Several people reported it and it quickly disappeared, whereupon apparently someone decided we shouldn't have nice things, or not for a little while at least.

        • stavros 3 days ago ago

          Yes but what was the incident like, in general? Was it just swastikas, or words too, or what? When I saw it, it was just normal fish.

          • ofjcihen 3 days ago ago

            I got you.

            So think of 4 Chan but fish. Like the entire screen.

            50 of the fish are just regular fish with slurs written in them.

            There’s like 10 swastika fish moseying around.

            And then you have the odd fish with like an Isreali flag with…let’s say stereotypical features for the face.

            • stavros 3 days ago ago

              Hahaha that is an excellent description, well done. It's as if I was there.

          • throwanem 3 days ago ago

            Quoting from the article's third paragraph:

            > But if you had the displeasure of viewing my website between the hours of 2AM (20 minutes after I went to sleep) and 8AM (when I woke up) EST on Aug 3, then you would have seen chaos. Every single username was transformed to a heinous slur, many unsavory fish had made it into the fishtank, and many beautiful fish were gone.

            At some point, asking for more starts to seem like rubbernecking at a car crash, you know?

  • vicdemydov1 4 days ago ago

    Great post-mortem, especially since it's a vibe-coded app.

    Curious if you were inspired by Lego's build-a-fish* exhibit at the Lego House? I visited recently and it is ridiculously addictive to see a fish you create swim with others :)

    https://www.youtube.com/watch?v=KYs3ne0HCwM

    • hallak 4 days ago ago

      Oh! I hadn't seen this. I was more inspired by the St Louis aquarium (where you color a fish and it swims) and Google's Quickdraw (a memory from like 2016)

      • XenophileJKO 3 days ago ago

        TeamLabs also has this in both of their main Tokyo art installations (Borderless & Planets).

  • Havoc 4 days ago ago

    Wild that some random used a security hole to try and counter the malicious actions actively lol

    • bombcar 4 days ago ago

      This has happened a number of times that I remember - one was a worm/exploit that would patch the hole.

    • robotnikman 4 days ago ago

      I remember reading about 5 or so years ago that the FBI was doing exactly this to counter the EternalBlue exploit (I might be getting the exploit name wrong)

    • bobson381 4 days ago ago

      this was awesome. people are surprising

      • er4hn 3 days ago ago

        There's a long history of this. A defense against the Morris worm made use of this as well.

  • amiga386 4 days ago ago

    The "S" in "vibe-coded" stands for security.

    • fourseventy 3 days ago ago

      The S is for snake... errr.. dragon...

      • bombcar 3 days ago ago

        I said more competent Vs!

        • rnrn 3 days ago ago

          *consummate Vs

          • DJBunnies 3 days ago ago

            He wouldn’t know majesty if it came up and bit him in the face.

  • richardw 3 days ago ago

    Got a killer original idea, got it built, learned a ton, is embarrassed about a few failures because he’s actually a solid professional. Well done, mate. If it didn’t cost you $100k, take down the network or lose you your job, you’re ahead and will laugh in a year.

  • SkyPuncher 3 days ago ago

    > It is really fun to just have high velocity, and it is really fun to not do code reviews and to just push stuff.

    This is so true. This post mortem also highlights a reason why so many of my side projects have died. There’s always a point where I need to get into the actual boring work of the project.

  • thehamkercat 4 days ago ago

    You can also upvote any fish without auth, limit is 20 votes per minute per IP

    POST https://fishes-be-571679687712.northamerica-northeast1.run.a... {"fishId":"xxxx","vote":"up"}

    • hallak 4 days ago ago

      That's actually intentional desgin - I think you can like a fish a little or like a fish a lot, and therefore should be able to upvote/downvote to your hearts content :)

      • pixl97 4 days ago ago

        The only potential risk I see with anonymous voting is some 4chaner is apt to create a bunch of fish with horrific names and use distributed IPs to upvote them to the top. I guess regular moderation could filter that out.

        • nijave 3 days ago ago

          IP based breaks users with CGNAT and people connecting from corporate networks

          In those cases you'd be denied votes since someone else used them up

          • VoidWhisperer 3 days ago ago

            The first case would be an issue yes, although this app doesn't exactly strike me as something that an insane amount of people would be connecting to from corporate networks

            • john01dav 3 days ago ago

              It can work okay for apps that don't go to massive scale and where users aren't concentrated in the same place. The chance of another simultaneous user on your block/building/CGNAT isn't extremely likely.

            • fragmede 3 days ago ago

              CGNAT is used in places other than corporate networks.

              • VoidWhisperer 3 days ago ago

                I'm aware of that - I spoke to the corporate network part specifically as the post I was replying to specifically listed that in addition to CGNAT

        • sdenton4 3 days ago ago

          Any button a human can touch is a vector for abuse...

      • lanyard-textile 3 days ago ago

        You write with such whimsy, absolutely delightful :)

  • js2 3 days ago ago

    There's a swastika fish on there right now. Folks get through the filter by putting it inside a fish shape:

    https://drawafish.com/rank.html?userId=1754341779700_log2xle...

    Edit: it's been deleted.

  • dmje 4 days ago ago

    Great project and a good post too, you write well and are funny. Would like to see more for my rss reader :-)

    • dmje 4 days ago ago

      …if you had RSS…

      • hallak 4 days ago ago

        I was 15 years old when they killed google reader, so can you blame me for forgetting about RSS?

        Added :) https://aldenhallak.com/blog/rss.xml

        • dmje 4 days ago ago

          Ha! Subscribed, and now the pressure is on to write something ;-)

  • somethingawful2 3 days ago ago

    > You may have heard differently, especially if you saw my doxxing[4b] on the unsavory website.

    Is it pretty common to get doxxed for getting to the top of HN?

    • thrance 3 days ago ago

      Just scroll to the bottom of this page and look at the idiots bragging about making swastika fishes. HN nurtures this community of mentally ill edgelords that we could very well do without.

      • rightbyte 3 days ago ago

        At my prior work we had this garage where we could wash our cars. But once the sponge was in the wrong place and the responsible foreman shut the whole thing down. "This is why we can't have nice things" he said.

        Just ignore the trolls and wait for the fish mods to ban them.

    • amiga386 3 days ago ago

      It is not.

      But if you show HN something... and it makes HN-reading KF users think "OH EXPLOITABLE!!!111!" so they post it to KF... they're basically going to test if your moderation works or not. If you only have some automated rules or even "AI" as the moderation, not humans, then they're very quickly going to work out what gets past your automated filter and what doesn't, faster than 4chan can make Microsoft's Tay say "Hitler did nothing wrong".

      Then the KF users will gloat about the slur-ridden fish they've made.

      Then KF-reading Sharty users will learn about it, the rest of the Sharty will join in the game.

      Then they'll get angry that your "normie" userbase has given a Palestine flag fish named "River to the Sea" the most upvotes, and an Israel flag fish the most downvotes, they'll want to cheat and hack their own creations to the top. So they'll start looking for an admin interface.

      When they find one, but find it needs credentials, the first place they'll go looking for password clues is all your socials, all your previous forums, basically your whole identity. And if they hit paydirt, of course they'll post whatever they find, because for them, doxxing people is fun too.

  • Retr0id 3 days ago ago

    From my (limited) experience poking at vibecoded apps, "broken/missing authentication" is the most common issue by far.

    That said, the 2021 OWASP Top 10 had "broken access control" in the top spot already, prior to the real takeoff of vibecoding: https://owasp.org/www-project-top-ten/ - curious to see the 2025 update.

  • davikr 4 days ago ago

    > I used the JWT to authorize login, but never confirmed that the JWT token belonged to the userId / email associated with it in the admin actions. So you could log in with my username and password, grab the JWT, and then send that along with your request.

    IANAWD: What is more appropriate than an admin token being able to authenticate admin actions?

    • alias_neo 3 days ago ago

      An admin token, that's presented by the admin it belongs to.

      It's like I have a security access card to gain entry to a building, it's not really serving its purpose if I give you my pass and you turn up, they need to check it belongs to the person presenting it.

  • lawlessone 4 days ago ago

    If you are the dev think you need to add an option to mirror the fish drawing.

    I can draw a fish facing left, but for some reason it's very difficult to draw one facing right.

    • busymom0 4 days ago ago

      Can't you just draw facing left and then flip it horizontally?

      • lawlessone 4 days ago ago

        Sorry, that's what i meant, i think it needs an option to flip it horizontally on the page, not mirror it.

      • rightbyte 3 days ago ago

        That is mirroring?

  • kergonath 4 days ago ago

    That was quite something. Thanks for all the fish. Also for posting this.

  • ChilledTonic 3 days ago ago

    I've noticed with nearly all of these "Vibe Code" security fatalities, they're nearly ALWAYS using Firebase as a backend. I get it, I've used Firebase for a number of enterprise and personal projects, its convenient and easy to setup.

    But even before LLM coding, I had team members walk into its numerous footguns - especially around public buckets and bad firestore rules. How many of these stories are really to be blamed on the AI tooling, and how many could be blamed on the very poor default settings of Firebase?

  • comrade1234 4 days ago ago

    I wish there some screenshots of the vandalism.

    Here in Zurich there's a mural of maybe twenty dinosaurs (not accurate but something that looks like it would be in a children's book). One day someone drew a dick on every single dinosaur. Even the flying pterodactyl had a big dick hanging off of him. It was so puerile and primitive it cracked everyone up that saw it. No tags. No football club graffiti. Just dicks everywhere. Thankfully the mural was repaired pretty quickly.

    • morkalork 4 days ago ago

      There's a screen shot of a certain site in the footnotes of the blog and if you visit it, you can probably find some screenshots in the 30-something page thread about OP's game.

      Edit: yes, the site too terrible to name as evidenced by the reply to this comment becoming dead within minutes l-o-l

      • jpk2f2 3 days ago ago

        I've never understood why everybody gets upset over New Zealand farmers.

  • HalcyonCowboy 2 days ago ago

    Brilliant write up, glad you made the effort.

    Trying to understand better where the JWT vulnerability was here, you said: “So you could log in with my username and password, grab the JWT, and then send that along with your request.”

    Am I understanding here that you weren’t validating the sub/userId or role in the JWT? I.e. any user with a valid JWT could hit the admin api endpoints? Or did I misunderstand that?

  • jmull 4 days ago ago

    On the allure of vide coding the author says,

    > It is really fun to just have high velocity, and it is really fun to not do code reviews and to just push stuff.

    Was slurfish fun?

    Looks like if you don't like doing deep and thorough code reviews, LLM-generated code is not for you.

    As the author concludes, "...LLMs are a tool. They let you generate a lot of code really fast...it is up to you to review it"

    • SirChud 3 days ago ago

      >Was slurfish fun?

      Yes.

    • morkalork 4 days ago ago

      >Was slurfish fun?

      It was for someone!

  • nullc 4 days ago ago

    I read the HN thread first, and the first comment I saw from the author was about AI nazi symbol detector they put in... I wonder how many orders of magnitude that comment increased interest in making offensive fish?

    Later I saw images of the attacked site posted elsewhere and thought they were both predictable and hilarious.

    • rightbyte 3 days ago ago

      A better group psychological approach might have been to only name the penis filter, ye. Then the transgressions are on the vandal who can't rationalize it is sticking it to the man, but that he is just that guy drawing swastikas.

      • nullc 3 days ago ago

        Or don't even mention it at all-- it's just got a fish detection filter.

        At least people trying to see if they can get around a fish detector are going to preferentially submit toilets and tires (and dicks, sure). :)

  • endymion-light 3 days ago ago

    I love the fact that you did a post mortem on a vibe-coded website

    Honestly - i think often so many people take tech very seriously so seeing this is quite refreshing and genuinely interesting from small side coded project point of view.

  • nullc 4 days ago ago

    So long and thanks for all the fish.

    • fnands 3 days ago ago

      So long and thanks for all the slurs? ;-)

  • bmau5 3 days ago ago

    This was so fun. Hope to see more like this

  • ofrzeta 4 days ago ago

    I am sure someone somewhere works on making LLMs commit code. Aside from that it was great witnessing the site in action and reading the postmortem. I wonder how the "hacker" made the connection to the user acount on neopets.com but maybe they just tried something like "ahallak"?

  • bikamonki 3 days ago ago

    Did I read Firebase? I assume you stayed on the free side? Or your attackers were mean enough to have you wake up to a five figure bill?

  • mmmlinux 4 days ago ago

    this is why we cant have anything nice.

  • whamlastxmas 4 days ago ago

    I was proud of my ability to have an extremely penis looking fish get past filters. I feel like when presented with censorship my instinct is to always test its limits

  • orta 3 days ago ago

    That was a fun read, nice work

  • gethly 4 days ago ago

    When i tried it, it almost blew my cpu.

  • koakuma-chan 3 days ago ago

    Do you ship stickers to Canada?

  • codingdave 4 days ago ago

    > I vibe coded...

    Well, there ya go.

    To be fair, it says the attention was unexpected, and this was just a coding exercise... And the port-mortem shows what I'd hope to see: digging in and figuring out root causes. So I'm not judging OP poorly over this.

    But still. Launching a vibe-coded app that accepts input from anonymous users is just asking for trouble. I'm frankly surprised it ran as long as it did without such problems. (Although I did see a few weenies swimming around even before the problem hours.)

    The lesson I'd pull from this is that if you are not the type of dev who could put together a post-mortem along these lines... don't launch a vibe-coded app.

    • famahar 3 days ago ago

      Ultimately an app like this caused more joy than harm. I'm all for people vibe coding fun little things like this when the stakes are low. Would prefer to see more non coders with diverse ideas feeling empowered to start a project rather than them seeing a huge wall to climb and never starting. The web needs more silly apps from silly people.

      • codingdave 3 days ago ago

        I agree with you to a point. But that point comes when the silly app turns into a channel for hate speech. At that point, I'm not sure we can justify the claim that it caused more joy than harm.

  • fHr 3 days ago ago

    nice one, great hobby project! great postmortem!