I in general find lay people have a very weak understanding of how research functions. This is a very generic statement, but everything from IRB processes to how science is reported in manuscripts and everything in between tends to be a quagmire, and this is absolutely with recognition that some of this process is mired in red-tape, bureaucracy, and endless administration.

For example, there’s a long-standing idea that IRBs are the gatekeepers of research. In reality, any IRB worth their weight (and really, all of them are for compliance) should be viewed as a research stakeholder. They should be there to make research happen and let scientists do the best research they can with the minimal amount of harm to participants. Sometimes this involves compromises, or finding alternatives that are less harmful, and this is a good thing.

Another common example is scientific studies are frequently criticized about sample sizes. Yes, a lot of research would definitely benefit from better sampling and larger samples, but narrowly focusing on sample sizes misses a lot of the other considerations taken for evaluating statistical power. For example, if one wants to know whether beheading people results in injuries incongruent with life, one doesn’t really need a large sample to come to this conclusion because the effect (size) is so large. Of course more numbers help, but past a point more numbers only add to the cost of the research without measurably improving the quality of the statistical inferences made.

In this topic about IRBs, A/B UI/UX testing for the set-up that Reddit did it, and being run out of an university setting? That’s hyperbole. I don’t like businesses doing aggressive user-focused testing without informing the user, particularly with UI/UX changes I dislike (looking at you too Twitch with your constant layout changes), but at the end of the day these kinds of testing generally don’t ever rise up to the threshold needed to be a particularly meaningful blip. Insinuating otherwise vastly mischaracterize how research is done in formal, structured settings.