Staging Ground is coming back and moving out of beta

First, great. I'm happy to see this come back. As a beta reviewer, I don't think I've ever felt more empowered to tailor a new question into a successful one.

There were a few standout points from when the SG's last beta round ended that developers for the SG recognized needed some thinking through. I made an experience report post on the Teams instance for beta reviewers and identified a number of them, and I also made a "From the Asker's POV" post there, which identified some potential pain points for askers. I'd like to resurface these problems as well as some other feedback and ask if you guys have come up with solutions for these areas.

I'm going to focus on the reviewer portions first, then the Asker's POV. I'm going to put a lot more effort into the Asker's POV feedback because I think that, if there's anything we need to tailor the Staging Ground to, it's the way the asker's experience feels.

Scale

Okay... Let's get the big one out of the way. I'm not going to mince words: The beta reviewers got burnt out. This lead to questions entering the Staging Ground, sitting in privacy for 24 hours, and then automatically graduating. That sucks for a question author.

What considerations have you guys made about scale? Stack Overflow already suffers from a lack of reviewer activity, how can we improve on this for the Staging Ground?

"I fixed it!" questions

When reviewing in the SG, I would semi-frequently come into a situation where I'd give some feedback about a question and submit it as requiring major changes. In response, the question asker would leave a comment and say that they found the solution (and, sometimes, even edit that solution into the question!)

This is a very tricky scenario. What sorts of solutions do we have for this? On the main site, I would edit the answer content out of the question body (if applicable) and encourage the user to post their solution as an answer (or, just flag/vote for closure if the question fits closure criteria). In the SG, that isn't an option, and I'm not going to hit the "Approve and publish" button just because the user found an answer... There are still issues with the question that could lead to it being closed on the main site!

Edits apply immediately at 500 reputation in the SG. Is that alright?

Anyone who reviews in the SG can have their edits applied immediately in the Staging Ground. Now... I personally love this. I'd honestly review on SO a lot more if I had full edit privileges on the condition that I have to be in the review queues to use them (hence why I reviewed in the Staging Ground so much), but I absolutely do recognize the danger that this can pose to a new user's question-asking experience if their question is edited by a new and potentially inexperienced user.

Has thought been put into the ramifications for allowing newer users full edit privileges when handling new questions? I'm not suggesting that you should change this, I'm merely bringing it up as a potential concern. The full edit privilege is available at a somewhat-high 2,000 reputation for a reason: It can be a bit destructive in the wrong hands.

Closure: Available at 500 rep and threshold is at 2 votes

The ability to vote to close is available to all SG reviewers and the closure threshold is at 2 votes (gold badge-wielders (when closing as a duplicate for their related tag badge) and moderators make it 1, still). This is quite the reduction in requirement for closure. That said... Reopening a question is quite easy for a question asker, as editing and submitting their question for re-evaluation will reopen it. (Don't worry though, reviewers: The option to decline re-evaluation will re-close it)

Is this reduced rep requirement and reduced close vote threshold something we're comfortable continuing with?

Asker's POV: "Major changes" review = comment replies instead of editing

This is taken from my "Askers POV" post. When an asker receives a "This question requires major changes before being published" review, they receive a notification. Clicking that notification (or following the link in the e-mail), they're taken to the reviewer's comment rather than prominently shown the feedback at the top of the post.

This causes a situation where focus is diverted from the top panel, which tells the user to edit their question in response to feedback, down to the comments section, where any reasonable person would assume they'd need to respond to that feedback with a reply, which is precisely what we don't want. I cannot emphasize enough how important it is to encourage users to edit their question instead of replying to reviewer comments, because it happens all the time (example is from when I was writing the "Askers POV" post).

Have some considerations been made to add some guardrails against replying to reviewer feedback in the form of comments, and instead encourage them to edit their question? My recommendation was to add a panel or tooltip of some kind when a question author begins writing a comment or comment-reply on a question that is in any state other than New or Re-evaluate.

Asker's POV: Could we get some more handholding?

There are a number of places where I think we need to hold askers' hands more.

The first point is right when you start asking a question that's going to be funneled into the Staging Ground. You get this fancy diagram:

That's great, but it crucially ignores an important step (y'know... The whole reason the Staging Ground exists?) - Responding to feedback! It doesn't mention that reviewers can ask for you to make changes and in order for you to get your question posted publicly, you've gotta make adjustments! My recommendation was to add an extra step:

Furthermore, when an asker posts their question, they're not given any information about what just happened.

I know, I know... We just told them what was going to happen, but I think we can do better. Could we add a panel at the top, reusing the ones you get when you receive "Major changes" feedback?

This does a lot for us:

• It explains what's just happened
• Explains what happens next
• It establishes that this space is where new information about the state of their question will be held

Lastly, when a question asker edits their question in response to a "Major changes required" review, they receive the following options when they go to save their edits:

This is confusing, because hitting "Save changes" doesn't prompt the user to submit for re-eval. They'd need to open the edit page again to be able to do so. Further, if you hit "Save changes", that helpful panel at the top is gone. Yaakov Ellis (a previous developer here at Stack Overflow who we do dearly miss) mentioned in a response to my experience report that the recommendation to simply have a "Save changes" button with the in-your-face option to submit for re-evaluation was under review (ticket number 794). Did anything come of that?

In all, I'm still quite happy to see that you guys are finally coming back around to new user onboarding and reviving the Staging Ground. I'd also like to take a second, as I do whenever I can, and sing the praises of all the developers (whether they're still working with Stack Overflow or not) who were involved with the Staging Ground project. Their constant response to our beta reviewer feedback was amazing. It often happened that bugs/feature requests beta reviewers gave were immediately responded to by developers, ticketed, and implemented within a couple days. That's fantastic, and I'd love to see more of it with future adjustments to the site.

How is AI going to be "leverage[d]" to scale this to all askers?

Staging Ground - a space for new askers

In 2022, we tested a concept in beta called Staging Ground—a sheltered environment where new askers could get help from experienced users to iterate on and improve their questions before posting to the broader community. This test was received positively by new askers and reviewers alike. We saw higher rates of successful questions making it to the platform. Askers felt supported and welcomed by more experienced users as they learned the ropes of how to ask a good question. Reviewers were happy to see their contributions lead to higher-quality content on the platform. It was a win-win for everyone. However, while the initial tests were positive, it wasn’t a scalable solution. Now, we can explore how to leverage AI to scale this feature to all askers—something that wasn't possible just two years ago. We are excited that we will be revisiting this concept and believe Staging Ground is a multifaceted approach to supporting community growth and content health on the platform.

Thank you

I want to take this opportunity to thank you for doing this.

From my experience as a beta tester, the Staging Ground can greatly improve questions that would otherwise just be downvoted and closed (which results in negative experiences of askers with SO).

There has been a lot of work put into the Staging Ground (both development work and beta-testing) and it would be a shame if that would go to waste. Thanks to the development team and everyone who was making this happen within the company (e.g., by getting other people to make it a priority). I am happy I had the privilege to beta test the Staging Ground.

Getting reviewers

That being said, I do worry about one thing. The Staging Ground needs a lot of reviewers in order to keep up with all of the questions. What are the exact plans on getting sufficiently many reviewers into the Staging Ground?

You mentioned badges in your post but what badges are planned? There were discussions about engagement of reviewers/scaling e.g. in here, this post and don't forget about that one (these links are only accessible to beta testers; sorry). Are there any plans on doing something about this? If so, please tell the Community (in public) what these plans are! Can we expect anything to happen in this area?

When?

Finally, what's the planned timeline? When will the Staging Ground be activated?

What will happen to the "First Questions" review queue? After all, it's pretty pointless for questions to first be reviewed in the Staging Ground and then go through "First Questions" (SG questions skipped that Review Queue during the betas).

Will it be deprecated/removed completely in favor off the Staging Ground?

Assuming it doesn't get removed/deactivated completely, if questions are automatically published due to nobody reviewing them, will these questions still go into the First Questions queue?

Will every question by new users go through Staging Ground? If yes, how do we get enough reviewers? If not, how will questions be selected for SG?

FWIW, during beta testing, a lot of questions went unreviewed which led to frustration for some users who expected immediate answers to their questions (I remember one comment asking how to bypass SG to get more eyes on their question).

A few thoughts about how Staging Ground needs to work to be effective:

Questions need to start closed

According to every description I've heard so far, questions that sit in the Staging Ground for a certain amount of time, and aren't explicitly acted upon to close or otherwise reject them, will simply pass through onto the site as is. This is unacceptable and misses the point of having this filter. The underlying presumption of the Staging Ground is that almost everything that would be submitted, at least needs work in order to actually meet the site's standards. (Daily reminder: the large majority of existing open questions do not meet the site's standards, and the site's overall quality greatly suffers for the fact that we did not get this in check a decade or more ago.)

If staff imagine that a default-permissive attitude will make for a friendlier user experience, they are being hopelessly naive. What will happen is that a huge fraction of questions will be ignored by the "experienced community members" (after all, they have little incentive to participate, and are few in number to begin with), or receive a single bit of feedback that is then ignored by the OP. The question ends up on the site, and then immediately gets downvoted and closed by the broader community. Or worse: a poor question that should be closed instead gets answered - except this doesn't even have the tiny value that it normally would, because OP has lost interest by now.

What needs to happen instead: Hold the question indefinitely until at least one person interacts with it; at that point, treat it as closed (and not upvoted), and subject to the usual "abandoned closed" Roomba rule. That is: if OP doesn't edit or interact for 9 days, delete it.

Keeping OP in the loop

I want to echo Spevacus here. It needs to be crystal-clear from the start that using the Staging Ground is interactive and that asking a question will potentially require multiple iterations of responding to feedback and not just receiving it. But going further than that, I don't think that trying to show a linear "flowchart" here is useful, and simply adding a step to it visually is a band-aid that largely misses the point. The existing UI:

comes across to me as window dressing that's insulting the reader's intelligence. Please instead use prose to explain the concept, and also set proper expectations - including the inescapable fact that some questions are fundamentally unsuitable for the site despite being nominally on topic. This can still be done in an upbeat, "promotional" manner - something like:

So you're ready to ask a question about programming? Great! Even the simplest questions can help contribute to our library - as long as they're on topic and well asked. The community is here to help make sure you have a suitable question and to refine it as much as possible, so that everyone can benefit.

If you have a question about a different topic, click here to see the other sites we offer on the Stack Exchange network. You can also visit the Help center to read more detailed information about the Staging Ground, as well as about Stack Overflow in general.

The next form will give step by step instructions on how to write a high quality question that's clear, focused, searchable and - most importantly - properly answerable. The better your question is to start, the more likely it is to be accepted onto the site and answered.

Once you submit your question, it will be reviewed by experienced community members. As long as the question is potentially useful, they'll be able to propose improvements so that the question and its answers can be useful for everyone. Some changes may be edits that the community proposes directly, while others may require additional information from you. Please respond to feedback as quickly as possible, to have the best possible chance of getting your question published.

Even if your question is found unsuitable, you might still get help in the form of someone pointing out a typo in the code, or an existing duplicate Q&A that answers your question.

Don't use comments!

The comment UI is far too clumsy for giving the kind of detailed feedback that most people need. It also leaves a massive clean-up headache, unless you're committed to automatically removing all comments when publishing a question out of the Staging Ground. (Seriously, the bulk of blatantly NLN comments on popular old questions is beyond absurd to begin with.)

To communicate properly about fixing questions, the OP and the "community experts" really need to be using chat, in nearly every case. (More than that, they need a fully-powered version of chat, that actually lets them workshop edits to the question and view them side-by-side with the chat going on.) Real-time interaction will keep the OP engaged - and allow for interjection when necessary (e.g. "please stop trying to post the code here; edit the question instead"). Finally, using chat means that the comment section on the posted question is clear, but there's still a reviewable transcript.

Aside from that, it needs to be much easier to highlight issues using canned comments as previously discussed. These should be templates, so that important details can be filled in; and they need to be far more comprehensive than simply generating one per standard close reason - as I already explained in my answer there.

This is training, not just for the OP

It's become abundantly clear that a huge fraction of long-time users don't actually understand the site at all, and are in fact actively incentivized to do things that are harmful to the site in the long run. The Staging Ground would be a massive help with training such users to close questions, to recognize issues with questions (i.e., what we consider to be issues) and just to have some expletive-deleted standards in general.

It would also mean that when people with the FGITW instinct get to see published questions (where the site permits answers), they're high enough quality on average that those answers aren't harming the site merely by existing. Heck, we could even incentivize them to participate in the Staging Ground by making it easier for them to get the first crack at those questions (by streamlining the UI, I mean, not by any explicit lock-out or anything).

Of course, this entails that there are systems in place to review Staging Ground feedback and approval actions. If there's one user who's constantly approving questions where others consistently find issues, that needs to be detected and handled.

It also entails that people need to feel empowered to help out with the Staging Ground. Anyone who wants to put on the "curator" hat needs an easy way to get access to multiple Staging Ground questions and choose the ones where they're best able to provide feedback (typically, due to having subject matter expertise). The queues don't work. The concept is flawed; the underlying principles of queueing theory would only make queues a good idea if a) every question needed to be deterministically handled in a "fair" amount of time and b) everyone reviewing could be treated as interchangeable. Neither is true.