44

Questions shown in the staging ground appear in the top questions and similar pages. I don't necessarily disagree with that as it could be a good way to fix the scaling issue in principle.

However, I have one big issue with this: People browsing questions (even with 500+ reputation and having asked questions before) don't know how the Staging Ground works.

For example, I have seen a user asking for clarification in the Staging Ground without using the Requires Major Changes option and asked them to use that option in the future. As a response I was told they were just browsing questions and didn't know about the UI. They also posted about this afterwards (thank you a lot for your insights). This user is by far not the only one with that issue bringing me to this request. Until now, I have messaged >10 reviewers about that via https://chat.stackoverflow.com. Out of these users, >6 responded. Most of them were saying it's related to the Staging Ground being new/not knowing how to work with it in some form or the other (and the other one was confused/stopped by the locking system). They just didn't know better and were thankful about being corrected.

Please, make sure that reviewers are explained how the Staging Ground works before they are reviewing their first question, commenting or closing it, especially for reviewers that get to the Staging Ground using the top questions page or similar.

For example, I could imagine users getting some information on how the Staging Ground works along with the reviewer guidelines when they view their first question in the Staging Ground.
I could imagine this being in the form with a modal shown when opening a Staging Ground post the first time. This should include:

  • What is the Staging Ground (a place where askers can get their questions improved before them being public)
  • How to review questions
    • This includes the reviewer guidelines or at least a link to it
  • Explain that reviewers should select an action (like "Requires Major Changes") when reviewing a post as opposed to just commenting
  • Tell (especially higher-rep) reviewers to prefer "Requires Major Changes" over closing. If a post can be improved by the author so it would be considered fine for the site, it should probably not be closed in the Staging Ground.

Alternatively, I could also imagine the Staging Ground not appearing in these question lists by default and only showing up there once the user received some onboarding by viewing https://stackoverflow.com/staging-ground once.
This way, we would still get a lot of reviewers seeing Staging Ground posts in the question list and these people understand how it works.

25
  • 13
    When I wrote Don't release and ignore it, this is the kind of issue I was talking about that shouldn't be ignored.
    – dan1st
    Commented Jun 5 at 11:19
  • 1
    Related: meta.stackoverflow.com/q/430513/6296561
    – Zoe - Save the data dump Mod
    Commented Jun 5 at 12:15
  • 8
    " When I wrote Don't release and ignore it, this is the kind of issue I was talking about that shouldn't be ignored. " - I mean... what did you expect? This is what SE does for a living. The SG was set up for failure the second onboarding was deprioritised in favour of implementing the SG first (or rather completely ignored; I doubt onboarding was ever considered). We don't have enough reviewers because there isn't other forms of onboarding, and we have bad reviews and lots of confusion around locks because the SG itself doesn't have onboarding. SE has already set the SG on a crash course
    – Zoe - Save the data dump Mod
    Commented Jun 5 at 12:18
  • 6
    See also meta.stackexchange.com/a/396939/332043 for a primer on how SE does product releases - the SG is joining the list of hit-and-run deployed features soon enough
    – Zoe - Save the data dump Mod
    Commented Jun 5 at 12:22
  • 3
    At some point someone has to wake up and realize: oh. People in the 21st century are lowest effort creatures. Maybe we shouldn't trust them to by their nature attempt to do the right thing.
    – Gimby
    Commented Jun 5 at 13:27
  • 1
    @Gimby but... waking up and realizing things would require effort....
    – Karl Knechtel
    Commented Jun 6 at 14:28
  • 5
    @Zoe "I doubt onboarding was ever considered". You are making it sound like you know what you are talking about, but you actually don't. It was very much considered, I was pushing hard for it while I was on the team, and had seen mockups and ideas in development around it. No idea where things stand right now though. And I don't think that you are a good judge of what can be considered ground for deeming the SG a failure, considering how many times you have done so throughout the history of the project.
    – Yaakov Ellis
    Commented Jun 13 at 6:02
  • 5
    @Zoe I agree that it can be improved. However (and I was not free to say this while I was on staff) I and the whole team found your repeated predictions of failure (like you do above, and have done many times throughout the project) to be incredibly unsupportive and very demoralizing. And so far, all of your predictions of failure have not come true. (1/2)
    – Yaakov Ellis
    Commented Jun 13 at 11:52
  • 5
    @Zoe (contd 2/2) This is 2 weeks after initial release. The team will hopefully get to more things, like a better onboarding, more reports, badges, and all the cool stats and leaderboards that were planned. All of these can continue to improve it. There are ways of giving feedback without the doom and gloom. I urge you to reconsider your tone. This has been continuing from the very first announcement of SG.
    – Yaakov Ellis
    Commented Jun 13 at 11:55
  • 1
    @YaakovEllis And you've continued missing my point since that very first announcement. I've tried multiple angles, and none of them have been good enough. At a point, I gave up. I'm once again pointing to my answer with hard numbers on the reviewer deficit that was blatantly ignored. That is why I believe the SG will inevitably fail. When the reviewer deficit was ignored in advance, even when faced with numbers indicating it, I have zero faith the features dealing with that will ever be implemented, when day 0 is the single (1/2)
    – Zoe - Save the data dump Mod
    Commented Jun 13 at 12:20
  • 2
    @Zoe your words were ignored because they were about scale, yet you kept at it again and again. And all of the work up to this point was testing and improving workflow, NOT scale. My argument was that the comparisons to RQ were not appropriate, since it was given no avenue for recruitment of reviewers, while SG will have much more visibility from questions in listings. Half of the items in RQ are ignored (give or take) and it is not a failure. I do not think it a stretch that auto-grad rates can be better than that in the long term, with a much bigger positive effect felt on the site.
    – Yaakov Ellis
    Commented Jun 13 at 12:25
  • 2
    Hi @Zoe and Yaakov maybe it would be better if you would continue this in chat? (Can't believe I'm saying this to a mod and former staff :P) Maybe this chat room?
    – Abdul Aziz Barkat
    Commented Jun 13 at 12:29
  • 3
    @Zoe The scale was of course a goal and it was of course designed for scale. Just that it wasnt actually tested before now. The beta was not about testing scale. Don't know any other way to put it. Scale is being tested now. And if there isn't the best onboarding in the first couple of weeks, it isn't the end of the world.
    – Yaakov Ellis
    Commented Jun 13 at 13:42
  • 4
    It may not be the end of the world but I do think the current lack of SG reviewer onboarding is somewhat of a big issue.
    – dan1st
    Commented Jun 13 at 13:45
  • 4
    I'll just weigh in at this point, @dan1st - we're on week 2. When we release a product in its early format, I don't think it's unusual that things like onboarding aren't included in the early release. That's stuff that can be built in, collectively with community members who have strong opinions about what it should say, over time. Nobody has forgotten onboarding, or ignored it. Nobody is pretending we don't need it. We just haven't gotten to it yet. If we are to release early and often, then sometimes those early releases will not be as feature rich as we would like for the long term.
    – Philippe StaffMod
    Commented Jun 13 at 15:26

2 Answers 2

Reset to default
9

We deployed two updates to work towards improving the onboarding experience for new reviewers.

Update 3 (Oct 22, 2024)

We've created the first onboarding video for reviewers. We'll update the help center article with a link and determine where else we can include this in-product.

Huge thanks to our product marketing and brand design teams for collaborating with us to create the video. We're working on a script for the next video focused on onboarding askers into Staging Ground.

Update 2

It was mentioned that new reviewers who see Staging Ground posts on the homepage/question list page may not understand what they should be doing.

As a result, their first interaction may not be going through to the landing page. To address this, we made an update to display the same onboarding illustration directly on the Staging Ground question page.

Update 1

We refreshed the onboarding illustration for new reviewers when they visit the https://stackoverflow.com/staging-ground/ landing page. Hopefully, the updated illustration and copy better aligns to 1) explain what Staging Ground is, 2) set expectations on how to interact with these posts (provide constructive feedback/guide new askers) and 3) understand the outcome reviewers are working towards (improving question quality)

reviewer onboarding illustration

Future updates

  • We are planning to create a short-form YouTube video (to support mixed media onboarding materials) to help onboard new reviewers that explains the "what", "why", and briefly how they should engage and interact (i.e. actions in the review panel)

As such, I will not be updating this post with yet.

11
  • Thank you for this update, this is (in my opinion) a huge improvement. Would it be possible to change the illustration to include something telling users to use "Requires Major Changes" for questions that could be improved/are not completely off-topic (including and especially but not limited to low-effort questions) in some way? See the "Tell (especially higher-rep) reviewers to prefer "Requires Major Changes" over closing. If a post can be improved by the author so it would be considered fine for the site, it should probably not be closed in the Staging Ground." point in my question.
    – dan1st
    Commented Jul 16 at 18:47
  • Yeah, hopefully/probably better Onboarding now, but the "Give guidance, but don’t give the answer" (only available as 7th item in the Help Center) should be more prominent... Maybe 50% of the Posts I checked and tried to review in the SG (I pick only Posts with Status = New), already had Comments in them (but no Review submitted) and the previous Reviewer(s) (often with 10k-25k Rep) was/were already answering the Question. As soon as sbd starts answering their Qt in Comments, the OP doesn't care anymore about any Change(s) required and only reacts to the Answerer(s).
    – chivracq
    Commented Jul 16 at 19:51
  • @dan1st thanks for the feedback, I'm leaning on keeping the illustration as broad and general for the purpose of focusing on explaining the "what" and the "why". I'll circle back with the designer to see what we can do for first time reviewers when they land on a question page for the reviewer panel.
    – tanj92 Staff
    Commented Jul 16 at 20:20
  • 1
    @chivracq we have some planned work to address the comment but no action applied issue. This is related to onboarding but is being treated separately in a different set of work.
    – tanj92 Staff
    Commented Jul 16 at 20:22
  • 1
    I think a video describing the most important things could also be useful with these issues (both that people should favor "Requires Major Changes" with a constructive comment over closing in many cases but also the thing about not giving the answer and probably also a few other things), especially if that video has good visibility together with the illustration/first visit page.
    – dan1st
    Commented Jul 16 at 20:33
  • Yep, we will incorporate somewhere where there is prominence as well as in the help center
    – tanj92 Staff
    Commented Jul 18 at 17:04
  • 1
    I wrote a MSO post on how to review questions in the Staging Ground, it may be a useful starting point for onboarding functionality/a future video.
    – dan1st
    Commented Oct 21 at 9:47
  • @dan1st Thank you for your contributions! Does this make sense to add to and expand the current reviewer guidelines help center article? I'll be updating my original answer with the reviewer YouTube video that was uploaded last week. We tried to keep the video as concise as possible and incorporating some of the things you mentioned in your original post.
    – tanj92 Staff
    Commented Oct 22 at 16:36
  • Takes quite a long time for the video to get to anything relevant to reviewers.
    – Kevin B
    Commented Oct 22 at 17:14
  • 1
    It might make sense to change the reviewer guidelines as there are a few things these don't mention which are still important for reviewing IMO but integrating everything from the post I wrote might be too much in addition to the existing reviewer guidelines. In any way, I think it might make sense to create a distinct MSO post for changing the help center.
    – dan1st
    Commented Oct 22 at 18:57
  • Looks like the 6-to-8 is closer to weeks this time, nice.
    – Karl Knechtel
    Commented Oct 22 at 23:46
4

Apparently, I was one of the commenters you tried to talk to and who didn't answer (I routinely ignore invitations to chat rooms). I have a personal rule for continuing to participate on the network: I only do what I enjoy doing. That leads to a secondary rule: No curation. I don't vote, I don't edit posts (except my own), and I certainly don't participate in Staging Ground.

I would appreciate an option to opt out of Staging Ground. Either don't show me any Staging Ground questions or show them with the normal question interface. I found it very annoying that I had to deal with a different interface because of something I don't want to participate in.

2
  • 4
    I wouldn't say it's necessary to use that option, I was mainly informing some users since I think most users are confused by this. If you don't want to see Staging Ground posts in the list, you can opt-out of it in your SO preferences
    – dan1st
    Commented Jun 17 at 7:10
  • OK. Thanks. I could swear I had looked for that but it seems I overlooked the setting.
    – Roland
    Commented Jun 17 at 7:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .