Thanks for chiming in! Several people reviewed the survey and the remarks you made didn't raise any flag. However there are issues you rightly identified.

I've got to chime in and second the point - a few of the questions were clearly phrased so that the answer would have had to be multiple choice.

Either rephrase them ("What is the medium *best for*", for example, rather than asking what they are good for, which could have been multiple things), or allow multiple choices.

You are right, using "best for" would have been more fortunate and avoided to conflict with the presupposition that the answer expected multiple results. That said, I don't see how this affect the quality in any way. Bearing the context in mind, the only way for participants to resolve the conflicting presupposition is to take the question to mean: "pick the best for you with regard to <aspect>".

Also, in some cases it said "You can skip this question", but then the "No answer" was already pre-selected.

Further, "No answer" is not the same as "Answering No" - I may not have an opinion on or enough understanding of something, not want to answer "No". Absolutely! When it asked me whether I was using other desktops, there was also no way to answer I'm not using anything else. I agree, again unfortunate wording. Should have caught that. But stats are not harmed yet, are they? Whether respondents picked 'no answer' or skipped the question altogether, we can still say: "Of the y people who took this question out of the z people who took the survey, x told us

I finished it, but I've concerns about the methodology and quality of survey results. I too have concerns, but my preferred solution (make all questions mandatory) will not please everyone and requires a lot of boilerplate (because of the 'no answers' everywhere). I understand the difficulty of keeping survey questions similar so they can be compared to previous ones, and also that designing good surveys is actually hard, but would suggest that in a next iteration, maybe this could be addressed? You won't have to wait for the next iteration: I'll be happy to revisit

That is indeed unfortunate. It's not good that there are two distinct ways of saying "I am not answering this one" (one by skipping, and the other by picking "No answer". That said, I don't think this affect the quality. When I am looking at the data, I see "x people took this question, of which y people picked 'no answer', and by the way z people took the survey." So the missing ys can be deduced from the xs and zs. The deduction is valid under the assumption that those who skipped did it intentionally, but I think there is an alert prompt that makes this assumption safe. As to why we sometimes included 'no answer', this is because there is no way to express "show question B if any (or none) of the answers under question A was picked". To get the existential, you need to make the question mandatory and give it a closed set of answers. However, we might have missed 1 or 2 times where the 'no answer' was not required, and the "you can skip" was enough. If so, I apologize for missing them. Off the top of my head I can't say for sure whether it could harm the stats, but I might be wrong. they were indeed using something beside an oS distro." That's a fine figure to work with. the survey's backend with you whenever you want and take note of any remaining issue.

Regards, Lars Thanks!

Adrien