Regarding the question "will it break too many posts?", what Adam Davis says is correct. So when we switch (and at this point I'm fairly certain that it's "when", not "if"), the most important thing to know is that this won't suddenly change old posts.
Here are some thoughts I currently have about the switch. I'm just thinking aloud; we haven't decided anything yet, so take it with a grain of salt.
Impact
For the vast majority of posts, it won't make any visible difference. The point of CommonMark is to break as few existing documents as possible. Yes, there will be posts that (purposefully or not) rely on edge cases that CommonMark handles differently than we currently do, and thus when an edit is made to an old post, some things may have to be manually changed.
That said, when comparing MarkdownSharp/PageDown behavior to CommonMark on a lot of Meta posts, there are three differences that impact a pretty big number of posts.
Two of them are continuation of list items and block quotes.
List items
Currently, this will create a list item with two paragraphs:
1. This is the first paragraph
And this is the second one.
With CommonMark (and even in a significant number of other Markdown implementations), the "second one" will not be part of the list item, but a stand-alone paragraph after the list. To make it part of the list item, you have to indent it to the same margin as the first paragraph like this:
1. This is the first paragraph
And this is the second one.
Block quotes
As for block quotes, at the moment when you write a post on Stack Exchange, this:
> Roses are red
> Violets are blue
will create a single blockquote with two paragraphs. CommonMark (and again, various other implementations) turns it into two separate blockquotes. To keep the paragraphs connected, the empty line needs a quote character as well:
> Roses are red
>
> Violets are blue
Headings
The third difference that sees a decent amount of posts being changed is with ATX headers like these:
# Introduction
## Impact ##
###Further Research
The third one will no longer work in CommonMark, because unlike what our current Markdown version does, CommonMark requires the space between the #
characters and the text.
Note that all three are related to typing as few as possible characters to achieve the desired formatting, which is probably why they are so common – why would you continue adding characters once you see in the preview that it looks as you want it to look?
Detection
My current thought is that we should, when someone edits a post that was created or last edited under the MarkdownSharp regime, check for these three issues and, if any of them is present in the post, offer to auto-correct them.
It is unfeasible to create a complete old-to-new converter and running it over all the posts, but at the time of editing (when it matters) I think those three major cases should be handled semi-automatically.
Diffs
This would be nice, but I haven't completely thought through all the implications and figured out solutions for all issues: I'm considering to keep MarkdownSharp running on the server for revision diffs (and suggested edits). So if you have a post that was created in 2014 but edited after the CommonMark switch, then for the purpose of showing the revision diff, we could render the old version with MarkdownSharp and the new version with CommonMark.
For some background info: Only the current version of a post is stored as a rendered version (see Adam Davis' point again); for older revisions we only store the Markdown source. The cached rendered version is what's displayed when you look at a question page, but once you click "edited by…" to go to the revision list, the diffs are created by rendering the revisions' Markdown sources on the fly and comparing the resulting HTML.
If you used CommonMark to render both the old and the new version, you would hide any major changes that were caused be the new Markdown engine and that should be handled.
For our incremental improvements to MarkdownSharp and PageDown, we have been living with that (we certainly don't want to keep each and every version around that has ever been used to render a post), but for the huge break that is the switch to CommonMark, it may make sense. (To be clear, there will be no option on question/answer submission to "render with the legacy Markdown version"; we would keep MarkdownSharp around only for diffs against old revisions).
Disclaimers
As I said above, nothing is set in stone yet. Everything above is just me thinking aloud. Also keep in mind that CommonMark is still evolving, so details in the spec may still change.
[link](foo(and(bar)))
something you find you need to do very often?