Staging Ground Reviewer Motivation

Why I generally do not review in SG:

1. Plenty of items do not go through it. Yes, I understand that is the idea. It still makes me feel that if I spent some time, it should better be on published questions because that would catch the ones that did not go through SG.
2. The standards of SG do not seem to match mine. I envisioned SG differently. Apparently, the aim is "barely adequate questions". So, things that need edits, or even clarification are supposed to be published. It reinforces my feeling that my time is better spent on published questions. Otherwise I would do double work.
3. I saw enough users who publish even non-adequate questions. Some do it because they do not know better and slipped up. Some seem to not really care because they just went through and approved a batch of SG items essentially at once.

Perhaps a more general problem with the entire design can be summed up with: Time is not infinite.

There are already a lot of areas where users spend their time: review queues, manually reviewing questions, asking, answering, editing, retagging, monitoring specific areas of the site, etc. Then Staging Ground is introduced and users are expected to invest even more time. But it cannot come from anywhere. Either they reduce their activity elsewhere on the site, or they take away time from off-site activities. Time investment is a zero sum game. We all have 24 hours a day. Reviewing in Staging Ground in particular, tends to take up more time per post than other activities.

Of course, I cannot speak for everybody. But at least for me, the required time investment without any meaningful reduction elsewhere makes me prefer not to. My specific points 1-3 are all about having to review newly posted questions anyway. If I have to do that regardless of spending any time on SG, then, I would rather just cut out the time I spend in SG.

TL;DR

I think introducing reputation in the Staging Ground is necessary but your proposal has some (quite significant) issues.

Rep in the Staging Ground

Thank you. As one of the people who participated in both beta tests, I'm convinced that reputation in the Staging Ground is an absolute necessity. I am completely in favor of the proposal to add reputation to the Staging Ground. I also think that most beta-testers (participating in the discussions about reputationin the staging Ground) and probably also many other active reviewers feel in a similar way.

However, there are still a few things that should be addressed (in my opinion). I mentioned some of these things already in the post on the Beta team but there are still a few things worth mentioning. Don't make the changes you are proposing as-is!

I don't expect you to implement all my suggestions but I'm asking you to think about and consider them as well as explain your choice.

Analysis on existing data

This is a significant change to the site and getting it wrong would be hard to undo. Considering that you now have quite a bit of data of the beta tests and also from reviews since the Staging Ground was released publicly, I am sure you did a lot of analysis (right?). Would it be possible to share the results of that with us? I'd be specifically interested in:

• What is the distribution of rep that would be gained by reviewers?
• Are there outliers (both with respect to users and questions) with respect to (simulated) gained reputation in the Staging Ground and how significant are these outliers?
• If someone would just approve questions in the Staging Ground (analyzing non-SG questions as if they were directly approved in the Staging Ground), would they get a net positive or negative reputation? If someone is approving every post they see, they should (in my opinion) not get any reputation out of it.
• Is there a significant difference between gained reputation with just an approval or after for posts approved after az least one author edit in response to "Major Changes"?

Too extreme values

The numbers you suggested are quite high. For example, you are suggesting to give reviewers 5 reputation per upvote which is a lot for a post they haven't written.

Use the score

I want to suggest using the post score instead of upvotes/downvotes. For example, if a post receives 1 upvote and 1 downvote, I don't think the reviewer should get reputation. I think it would be better if an ipvote and a downvote would neutralize in that regard.

Major Changes

The most important action in the Staging Ground is "Require Major Changes and this is also the option that takes most work for a reviewer. I suggest also giving some reputation to reviewers requesting changes (with appropriate limits TBD) if another reviewer approved the post (which happens quite often).

For example, you could give these reviewers some reputation for positive scores only (and take some of that away if the score decreases but make sure people don't lose reputation for this action (except for spam posts)).

answering should give more rep

Asking and answering are the main forms of gaining reputation and this should stay that way. Please make sure that typical users can get more reputation by answering questions than by reviewing Staging Ground posts (in comparison to the time it takes to do so).

Duplicates

Reviewers aren't supposed to be subject-matter experts. You can't expect them to find all duplicates. Please don't deduct reputation for posts being closed as a duplicate of another post. Nevermind, I didn't read the exclusion.

On the other hand, finding duplicates can also be a lot of work. What do you think about giving a small amount of reputation to the reviewer who originally found the duplicate if the asker agrees (either by directly agreeing or another reviewer using the same duplicate and the asker not Re-Evaluating). Reputation for duplicate finding could also be a good idea on the main site (see this comment).

giving a lot of reputation for staying open

You include giving 10 rep for

Question remains open on Stack Overflow and was not closed at any point within the first week

This seems excessive to me. I don't think there should be a (significant) reputation gain for posts that are not receiving certain actions. If it was like 1-2 reputation, I'd probably be ok with it but 10 reputation for every post that stays open is way too much.

What happened to the original proposal?

In the teams post you linked, there was a different proposal. Can you please explain why you made the changes you made?

The original proposal was (mostly) the following:

• +3 rep on upvoted posts, both when the reviewer approved or requested major changes
• -1 rep on downvoted posts you approved
• -5 rep for non-duplicate closures
• +1 rep for closing as off-topic (I agree with the removal of this)
• +2 rep for duplicate closure (I do think this could be done in a better way than in the original proposal which I mentioned in my suggestions there)

There was a lot of discussion on that proposal including a few problems that were noticed with it but overall, I think this original proposal was better than the current one so I would be interested in what exactly motivated your changes. Did you start over completely?

Giving the wrong incentives

Your current purposal of how to award/remove how much reputation seems to incentivize any reviews, no matter how bad they are. It looks like it would award a lot of reputation even to users that deliberately approve bad posts just because many of their reviewed posts stay open.

Here are some things thah should (in my opinion) be incentivized:

• Using "Requires Major Changes" in a way that improves the post
• Finding Duplicates
• Approving good posts

And the following should NOT be incentivized:

• Approving any post without looking into these posts
• Other forms of Robo-Reviewing

If users who are approving like 15 posts in 10 minutes (especially when they aren't skipping many posts) get a net positive out of this, you probably have the wrong system.

Abuse

With any way of introducing reputation to a system like the Staging Ground, there will be some people (trying to) abuse it. Your proposed system with these extreme values for questions staying open and upvotes is especially bad as people would just get reputation for many reviews, no matter how bad these reviews are which (I think) makes it very easy to abuse.

Before making any system that introduces reputation incentives to the Staging Ground, make sure you have good abuse prevention and detection mechanisms in place and ask the moderators whether these mechanisms are sufficient and what else is needed for that. If moderators tell you the mechanisms you built are insufficient, listen to them.

Aside from building the said mechanisms, make sure you are ready to quickly build new ones after reputation in the Staging Ground is available depending on the exact needs.

Other things

Providing incentives to review questions is one (important) thing but it's also important to make sure these reviewers know what they are supposed to do. Please provide an onboarding experience to reviews that:

• shows them example posts and how to review these
• tells the reviewers what is expected from them, how the review options work and what a good post looks like
• they can view again at any point

Who does a reputation reward work best for? Same question for badges?

I personally feel both of these rewards are great at attracting attention from newer users, but have little to no affect on experienced users (or even diminishes it in some cases.)

I think a much better tool for encouraging participation would be making it clear what positive effect our contributions are having on the network at large. Provide good reporting on the SG that everyone can access including users who can't review so they can learn to trust the process.

The entire point of this platform is being the place where you can find answers. While I think the staging ground is important and the work that it is doing is needed, I also feel its goals are misplaced. It shouldn't be a practice of how many questions can we get approved/posted, it should be process of preventing posts that will not fair well from being published. All of the rep incentives work against that.

Don't exclude “needs details or clarity” from closure penalties

For approving questions that are later closed, you have “needs details or clarity” as one of the exclusions from the -5 rep penalty. It's not totally clear what the motivation for that is, but there are a few problems with that:

1. It's often used on stuff that's totally incomprehensible, wildly underexplained, using images of text, etc.. Consider this example from SG (reviewed as requiring major changes), or this example from the main site (screenshot for <10k; closed as "needs details or clarity"). It doesn't take a subject-matter expert to identify that these are totally unsuitable.

2. It's nearly interchangeable with "Needs debugging details" (other than that "Needs debugging details" can't reasonably apply to how-to questions), which does not have that exclusion. I don't think that there's a clear reason that one should be excluded, but not the other. If anything, it can be harder for non–subject-matter experts to identify whether a minimal reproducible example is adequate.

I think that the main problem with Staging Ground and lack of reviewers is that there is not enough questions in the SG to begin with.

For providing adequate guidance in the SG, or what I think it would be adequate guidance that would ensure success of the question once published is that user giving guidance needs to be subject matter expert, or at least knowledgeable enough.

I can cast close vote on obviously poor question that lacks details, asks multiple questions, lacks reproducible example and hope that once closed OP can follow the guidelines in the post close notice and read more about asking in Help center.

But in SG I cannot provide sufficient guidance, except for very basic one in line with the guidance that can be found in canned comments, if I don't know anything about that tech stack. Leaving such basic comments and hints (which we must give in SG) that can only address some issues in the question often results with OP making some changes and then replying to your comment asking for further feedback or hoping that question can be published, but at that point I may not be able to provide adequate feedback anymore. And I may not be in position to tell whether question is ready for publishing or not.

There are plenty of users who frequent various tags and post comments and give feedback on numerous posts in the area of their expertise. But they simply don't get to review SG questions, because there aren't any or very small number of those.

Instead of figuring out how to give reputation for reviewers, you should try applying SG on large scale first. That would be the most appropriate test for the whole SG. That way you could get more feedback about the workflow and potential problems and possible improvements from users that are commonly doing curation work in their area of expertise.

The worst that could happen is that poorly asked and researched questions will not be instantly answered and that if they eventually get auto published because they were not reviewed, that posting delay will discourage people from asking without putting some effort first. And those askers that put some effort will get rewarded as their questions will be better received once published.

This would also prevent new users to ask really bad or several poor questions in a row getting an almost instant question ban after joining the site.

I'll mostly repeat what I said in the beta SO for Teams site.

I don't think this should reward rep. Precedent is relatively weak, and robo-reviewers are already enough of a problem in review queues where all they can get is badges.

If it must be rewarded rep, the proposed values are too lucrative. I propose maximum absolute values of 2, except that publishing spam or R/A should be a review ban and something harsher like -50 rep.

From an abuse-case perspective, you are essentially creating a proxy channel to perform serial-voting on reviewers (assuming SG reviews are or eventually visible in a user's profile page- which they should be). Ex. say I want to act on my pettiness and have something against reviewer A. I can downvote everything reviewer A approves and make their rep go down. I don't see what's so difficult about figuring out what kind of mod tooling you need to detect rep-based abuse- at least at the first order of the most obvious abuse-case this will enable. Integrate the review system into the existing serial-vote-detection system. As in- in that system, treat a vote by user B on post approved by user A as a vote by B on a post by A.

Rep gained is locked in after 14 days to protect against long-tail vandalism.

How is this a net improvement? You want to know what thinking "long game" looks like? "I can make a voting sock, very slowly upvote posts reviewed by my other account, wait for 14 days to pass since each vote, edit the post, and then remove the sock upvote to cover my tracks. Receiver account keeps the rep, but the vote is gone.". Unless mod tooling keeps tracks of votes as transactions in addition to the latest existence of a vote by a user on a post, and factors that into fraud detection, and has parameters wide enough to catch it, that's going under the radar.

We are trying something new, and this is just our first attempt at dipping our toe in the waters of rewarding curators with rep, a form of recognition which we believe is overdue

Why this specifically, then? See for example, https://meta.stackexchange.com/a/387398/997587.

In future iterations, we are open to the possibility of allowing multiple reviewers to gain rep from the same Staging Ground post, rather than just the final approver, as this is often a collaborative process. - Sasha

I can get the motivation (I actually mostly look at just new stuff in SG), but it's a nested can of worms in what's already a can of worms. I can imagine people abusing it in similar ways to how you can abuse "leaving a review" in First Questions by just upvoting a comment. Except now they are doing it to get rep and not just a badge. And what if I only review new stuff, and other people publish something in a state that I don't think is ready? Would you penalize someone for a downvoted post in that scenario? It's messy.

I'm an active SG participant and participated in the SG betas.

Unfortunately, at this point, these numbers are not growing.

Not enough new posts that ought to go through SG first are fed through the system.

Staging Ground, however, has one small problem: it takes a lot of work. If It succeeds, it will only be on the backs of community members, like many of you, who work to make it a success. However, even this is not enough — a lesson we’ve learned time and again (for instance, with review queues). For it to succeed, there must be enough reviewers over a sustained period to meet the number of questions coming in.

Is this really a problem though? SG posts that receive no interaction are eventually auto-approved and published to the main site where they are handled through normal means anyway.

In order to make Staging Ground sustainable, keeping existing reviewers engaged and attracting new reviewers is a priority. In addition to adding more badges for reviewers to earn, the possibility of earning rep is a potential incentive to increase reviewer engagement.

I can't really agree with the gamification aspect of SG reviews. With gamification comes rep farmers which comes with audits which leads back to the review queue issues. The review queue audits every 3rd to 5th review are the reason I quit the review queues. I'm not saying audits will come, but there is history here. I can certainly say if review audits were introduced to the Staging Grounds that my participation would be considerably less.

I think recognition in the form of earned badges to highlight a user's contribution specifically to SG could be acceptable though.

I could get behind a rep loss/penalty for SG post reviewers that approve posts that are immediately down-voted and/or closed on the main site. The entire point of the SG is that posts have been peer-reviewed and meet the site's standards before being published. Quality should be the bar, not quantity.

Bold ideas, and I like the badges, mostly (the naming leaves a bit to be desired, IMHO).

However, I think the suggested reputation table is far too generous to participants; it rewards secondary efforts as much as or more than original efforts. Here is my suggestion for a more measured reputation reward table:

Additionally, similar to the Suggested Edits review queue, there should be a "global" reputation cap for Staging Ground reputation: no more than 1000 reputation should be awarded.

Likewise users should not be penalized more than 500 reputation for negative results. Instead, users should be permanently suspended from the Staging Ground if they reach that point. Auto-suspensions should also probably be implemented at intervals of 100 reputation lost and 250 reputation lost, for 1 month and 6 months, respectively.

Notes:

• I combined the two "gain" rows into one; I think the notion of giving someone 10 reputation just because they read a post and click a button is... excessive.

• Approval alone does not merit any reputation reward, so that option has been removed from the remaining "gain" row.

Just to sum up on how the current proposal's reputation gains may work in practice..

A reviewer who approves, say, 35 reviews a day (looking at a real reviewer who approved 35 posts within 2 hours in the past day,) if the posts they approved remain open for 7 days they will earn 200 rep 7 days from today.

If 10 of those posts were closed they will have still earned 200 rep 7 days from today, and they get to keep that rep regardless of whether it gets closed some time after that 7 days.

This isn't taking into account any rep they will earn from the first 5 upvotes that post receives at any time in the future. As of right now, only one of their approved posts is closed because the post wasn't in english... proving they either were just robo reviewing, or don't have a firm grasp on what posts should be approved. None have been downvoted, several have been answered. Yet, the more reviews they approve at the current reward/penalty rates, with the rep cap, the less likely they are to miss out on the rep cap 1 week from now from just quickly approving a bunch of stuff. Granted, this is just one user, and I see no evidence they repeated this, but if they earned rep from it... surely they would? I do see other users who have likely reviewed in this way as well, all before a rep reward and badges even exist.

Now, we could certainly expect mods to police this and curators to flag this behavior, but... in practice is that going to happen? That's certainly not being incentivized.. Are we instead going to get an annoying audit system that gets it wrong half the time? Or... just take the increased questions being approved as proof of the incentives working.

Staging Ground's impending failure is merely a microcosm of the curation problem that Stack Overflow has as a whole. And that problem is simply that there is quite literally nothing, short of paying actual money, that you can do to incentivise people to increase their curation activities.

The reason for this is the one that the community has been warning Stack Exchange Inc. about for years: question quality, or rather the lack thereof. Nobody, and I mean nobody, wants to wade through oceans of sand in order to maybe find a pearl - because there are so few pearls and so much ever-increasing sand, that no amount of fake internet points you could get as a reward for finding a pearl could ever be worth the effort.

I don't even know why I'm writing this answer because you've consistently demonstrated that you have no intention of honestly addressing this fundamental underlying problem. Until you do, these band-aid initiatives like Staging Ground will continue to accomplish nothing and you'll continue to wonder why. So yeah, go ahead and award more rep for participation there. Go ahead and add more badges. Go ahead and do whatever you tell yourselves will make the Staging Ground line go up, because the latter is clearly the only thing that you actually care about.

I'm just looking forward to seeing a staff post in six months or a year, "Driving Staging Ground engagement has caused <new problem> that we want to fix with <new solution>". I'm sure y'all will be exactly as receptive to community suggestions then as you have been in the past, just as I'm sure that when your new solution causes a new problem you'll be ready to tackle the latter head-on.

After all, lines need to go up, right? Even if those lines are "problems we've inflicted on ourselves" and nobody sane would ever consider that a measure of success. But I'm sure you'll manage to convince yourselves once again that the definition of insanity isn't, in fact, doing the same thing multiple times while expecting different results.

Imaginary internet points are not a sustainable reward mechanism for activities that require time and expertise. Curation isn't fun. I will solve protein folding puzzles all day long without compensation because I love a puzzle and I'm advancing human knowledge (or was until AI started doing it better lol, but I probably helped get it there). Trying to get some stranger's question into shape when they are just as likely to get upset about it as appreciate my effort, with very little recognition from my peers or the company, when there will always be more questions that need curating no matter how hard I work at it, is not something I'm willing to do for free, especially now that the data dumps aren't publicly available.

Curating on Stack Exchange is work that requires not only expertise in the topic and English fluency but also expert knowledge of Stack Exchange rules and community norms. There is no reality where reputation points will be enough reward for most people in the current SO environment.

When are humans willing to do thankless work without getting paid? When it's fun. When it's directly in service of a cause greater than lining corporate coffers. When they are able to connect with the people they're helping. When they form a community and connect with other helpers.

I only see two options, because curation will never be fun for most people. Pay people in some sort of proxy currency or go back to Stack Overflow's origins where a small core group of dedicated people work hard to do something for the good of everyone with the company as a partner. Since it's really hard for the part of the company who makes things to change the part of the company who's just interested in making money, and a small core of people working really hard doesn't scale well without a significant investment in technology, I think the only viable option is to give people the opportunity to earn something of tangible value by curating.

Let people earn credits to use the AI or other paid services you offer. Let them earn extended profiles. I don't know what would work best. The problem with reputation is that it's not good for much once you've earned a certain amount. If people are going to work for the company's benefit, the company should compensate them with something that has congruent value.

I'd love to see some stats surrounding how many people with posts in the SG are responding to feedback within an hour/day/other. I just tried looking through the first page of SG questions that are tagged with python and I could only see one that had any semblance of an update.

This was only a small sample but I wonder if a larger one would give more favourable results. No amount of internet points would make it worthwhile if it turns out review feedback is falling on deaf ears

This proposal bakes in other limitations such as:

Reviewers are still subject to the existing rep cap — no more than 200 rep earned per day.

There's a 1000 rep cap for suggesting edits, see Why is there a limit on suggested edit reputation reward?.

In this proposal you make no mention of any all-time rep cap for SG, just the 200 daily rep cap. That means what you're aiming at is Bodies in Seats. That means what you want is as many reviewers as you can get in exchange for imaginary internet points (changing what reputation is) to spend as much time as they can loose regardless if it's unhealty and promotes internet addiction (something that's been mentioned quite a few times about the fanatic badge).

So no, an unhealty uncapped internet addiction promoting gamification isn't something I support. Put an all-time cap on it.

My personal opinion
For me personally, the risk of actually losing reputation for trying to help new members would just make me stay away from the SG all together. I haven't contributed much to it, simply because I'm not always confident in my judgement. And I don't want anyone potentially suffering the consequences of that. But being punished for doing my best certainly doesn't encourage me at all.

Issues

• I have no control over what happens with a post after it's posted on SO. If the OP decides to completely change their question and it results in a lot of downvotes / deletion, the reviewer(s) are to suffer for it. I know there is the question's edit history to fix this, but that would just add a lot of workload to moderators.
• I have no control over votes either. People can down- or upvote content for any reason they like. It'd be unfair to a reviewer if they lose reputation on a question that's been downvoted just because someone didn't like OP's profile picture.

A better approach (in my opinion)
Reviewers should earn the right to earn reputation for reviewing. Have certain badges in place that are required to be eligible for reputation gain by reviewing. Something like a number of successful approvals that remained on SO with a positive reputation for 1+ week. This should prevent most people from trying to abuse the SG to earn reputation. They'd have to prove themselves capable of properly reviewing content first.

Similarly, you could revoke a badge (or whatever method you wish to use to make someone eligible for reputation gain through reviewing) if too many approved contents end up downvoted / deleted.

Using this approach, you don't end up (unfairly) punishing reviewers. You still encourage them to become a reviewer. And you discourage abuse.

My thoughts:

• Asker only badges will be unobtainable for users already beyond the threshold to be no longer eligible for staging ground, unless you do questionable things like make a new account to earn the badges and then merge them.

• I am not sure if downvotes on questions should penalize the users who approved them. Because what about 'not useful' questions that are nevertheless on-topic and clear? What alternative would there be aside from someone either tanking the penalty and approving them, or leaving them in the Staging Ground limbo for ever?

  I do however agree with a rep penalty for off-topic that get closed, or spam/abusive questions, because those are posts that should absolutely should not be published.

• I agree with other answers that state that the rep gains and penalties are too extreme. Perhaps only +2 for posts that get upvoted, and -2 for posts that get closed. The primary source of reputation should still be generating useful content yourself.

• Will these reputation changes be awarded retroactively? Based on the wording the implication is no, but I am not sure for a fact.

While your proposal incentivizes making review actions (specifically approvals when it comes to rep), it does nothing for actually improving posts (or letting the asker do so).

While I already gave you feedback on the reputation incentives, I want to suggest a new badge in this answer.

I would like to see a badge (or multiple badges with differentratities) that is rewarded when a user reviews at least N questions as "Requires Major Changes" that later receive and author edit and are afterwards approved (by any reviewer) without any edits (maybe excluding the author) after publishing the post for 48 hours.

The value of N would need to be chosen appropriately, maybe 10 for a bronze badge and 50 for a silver badge.

Adding such a badge would provide an incentive to

• ask the author to improve their posts if these posts are missing something
• edit the post if there are thingss that should be changed (spelling, formatting, title, tags, whatever)
• make sure approved posts have everything necessary

You could also make the rarest version of this badge be awarded multiple times to encourage this behavior also to users who already earnt these badges.

Why do we need to track the progress of reaching 50 reviews in one day as a badge when we already have a progress indicator right there at the top of the right column constantly reminding us that we haven't done our share of work?

On that note, how much time are we expecting users to generally spend on a review on average? Should we even be showing the upper limit of reviews in this way, as if it's a daily goal, despite it being something most reasonable people reviewing for the right reasons probably won't reach because reviewing in this way actually takes a decent amount of time?

I've not worked out how the code editor in the staging ground reviews works. AFAICS, it's completely different from that used in the main SO interface. I assume some people can make it work, but I can't edit the code in questions using it. This is, IMO, a major disincentive (demotivator) for reviewing staging ground questions.

The number of unique reviewers, the average number of reviewers per day, the adoption rate (the number of users who actually became reviewers, out of all the eligible active users who have access), and the number of questions reviewed per reviewer. Unfortunately, at this point, these numbers are not growing.

These seem like extremely incorrect metrics. Maximizing the number of unique reviewers is silly--if most users have absolutely no interest in helping new users, they shouldn't have to. And they probably wouldn't do a good job at it if forced to, either.

Instead, you probably want to cater to the smaller portion of users who do enjoy being helpful to new users. (Thankfully I think the changes proposed here are fairly in line with keeping those users engaged.)

It seems like the key performance indicators for reviewer engagement are:

• Average time to first response: If there's too many questions and not enough reviewers, this will be longer. If there's enough reviewers for the question rate, it'll stay the same. If people have to wait for 2 days for somebody to even tell them that their question is bad, they won't return to StackOverflow. Meanwhile, if they get quick feedback and then get approved, they're more likely to return.
• Queue length/fall-through rate: If there are too many questions and not enough reviewers, the number of active questions will grow until it can't keep up. If an increasing number of questions are not getting responses and getting posted on the main site, that's a problem.

The overall topic of these proposals is to reward curators with rep/badges in order to increase participation of reviewers in the staging ground. Surely, in the beginning there was a novelty effect of the staging ground, but by now it's the real performance. However, curators not only work in the staging ground but also outside of it. Badges are also given out for curation actions outside of the ground, but rep isn't (except for +2rep on edits for users with less than 2k total rep). Sometimes even rep is taken away (for downvotes for example). Also in this proposal, negative rep is proposed for certain actions.

Your goal is to increase motivation of reviewers but also to steer their behavior by gamification. It worked in the past, so it probably also will work in the future. However, the extent of how much you can increase the motivation might be limited. Not all people value badges and reputation that much. It might not be enough, or both goals might counteract each other. For example the -20 rep hit for question that are approved but later get deleted might have a significantly chilling effect on motivation because mistakes (can happen) are punished too much.

My personal motivation is torn. On the one hand, I think that the staging ground is what was always missing on this Q&A platform. Questions should only allow answers when they are answerable and this should be checked before, not at the same time with everything else. On the other hand there are backsides too (good questions have to wait longer and it is a lot of additional work and there needs to be done additional checks and decisions for every question before it even gets answers). Curation surely is hard work and should be rewarded but I still don't care about rep and rep mingles everything together anyway. Also in general, my motivation to contribute to the platform is low, and for whatever reasons, maybe the good idea of the staging ground will simply die because of a lack of curators willing to do the work in the end. Doesn't make it a bad idea though, just an unreachable goal.

One thing that I would really like to know: how often is advice in the staging ground ignored and how often is it actually taken into account and results in a better, answerable question? If for example I was a staging ground reviewer and I would feel that I'm annoying people rather than helping them, I would not want to continue probably. If curators do try their best, are question askers doing their job too or not?

Otherwise, streamline, streamline, streamline. It should be an as smooth as possible experience for everyone. Do comments really have to deleted upon approval? Are automatic up votes upon approval a good idea? Can it be integrated better with the "normal" ground? Make it easy to discover a sufficient amount of fitting questions for reviewers, make giving feedback as quick as possible, try to decide things as automatic as possible (for example a 5 character edit hardly ever solves a serious problem in a question, or so). That should help those that discover the staging ground to actually work in it for a longer period.

Sorry for not being able to go into greater detail.

For badges that are awarded multiple times based on stat counts, how is it counted?

• If I have a total of two "variety" badges, does that mean the total count of distinct tags over all the SG items I have ever reviewed is 100-149?
• Is it possible to get two "steadfast" badges by reviewing 8 days in a row? (two distinct overlapping windows of 7 days in a row).

We need off-topic reasons for the usual no debugging details, or too broad on the SG would be helpful. Since it's the stuff that new users need the most. Having none of them would let the reviewer type different things for every vote or copy any template they had that might not be correct.

I think SG is cool concept, However it adds another task for the community which is already moderating quite actively by flagging and editing. So given that SG does not add much value. And if we introduce reputation as an award it'll take away focus from other queues.