Tool for automatically correcting indentation and formatting of CR & SO code

This is my tentative solution to:

How can I prepare my code so that I can paste it formatted?

If you need to copy and paste code into the question or answer textarea when solving someone's problem, describing your own, or fixing someone else's formatting, this tool will correct the indentation of the code and prepare it as a code block as understood by the Stack Exchange formatter. Pending CR's improvements, it will some day hatch into a bookmarklet, and none of us will hit the space bar repeatedly or alt-tab between browser and IDE ever again.

In the interest of having a great thing that can make all of our CRing easier, I invite you to review this without mercy. Readability, usefulness, you prefer .concat to +, you just stubbed your toe - you have a gripe, let's hear it.

The following were my concerns while writing this, upon which I would specifically request comment:

• I'm pretty sure the way I'm counting delimiters is not the most obvious route, but it works in what I think is linear time. Is it difficult to follow? I kept tossing out comments thinking I wasn't getting the whole strategy across right.
• Am I overdoing map, filter, reduce, etc? To me, it's easier to see the intent of an iterative process and to analyze its performance characteristics when I use those methods than using while and for for everything. But I don't see a lot of code that uses them as much as I do.
• Clearly there was some iterative factoring going on, and not all methods have been neatly bundled into classes, so I invite comment on a more organized set of abstractions, or just how you would have factored it differently.

There are also some features which are just not going to fit into what I've got right now, so I would like input on how you think their absence impacts the effectiveness of the tool and how you might go about implementing them without a total re-write and/or moving on to a grammar-based solution. These are:

• Can't enforce token parity (e.g. doesn't care if it sees {)). It's not impossible to do with what I have, but I think it would affect performance (requiring a stack or recursion instead of just counting) and not generally be worth it.
• Doesn't see any difference between grammatical contexts, e.g. "{" and /*{*/ will both cause indent. I think it would be stupid to try to do this without a grammar, but it would need a separate grammar for every language it would support. It's already one monster of a bookmarklet.
• It isn't going to do hanging indents for cases, public:, private:, or anything else where a single token dedents the line it's on but indents the next line. It would have to be split up into two tokens, and would probably need to be tweaked per-language. Or I would need to do grammars.
• It isn't going to do hanging indents for statements missing a semicolon. A regex can't determine what statements need a semicolon and which don't.

It was originally going to have a "just turn the tabs into spaces" mode, which is why you see it messing with tab/space alignment. Is this a feature that would be useful? It would take trivial effort to implement.

Finally, I want to hear what additional features you might want, and what languages you want added (optimally along with a token set as described by the code).

Fiddle with it here.

In the fiddle, the "bookmarklet" runs on page load, adding the toolbar beneath the textarea.

(function ($, config) { "use strict"; var // this regex digests a string into leading whitespace, // content text, and trailing whitespace. lineRegex = /^(\s*)(\S.*\S|\S)?(\s*)$/g,
spacesPerTab = config.spacesPerTab,
// digests a string in which sequences of spaces equivalent
// to a tab have all been replaced with actual tabs.
// remaining spaces that are not trailing all tabs have no
// effect on alignment.
aligningTabRegex = /^(\s*)([^\t]*)/g, adp = Array.prototype, predefinedLanguages = {}, MicroToolbar, repeatSpaces; repeatSpaces = (function () { // we'll be repeating spaces a lot, so memoize and generate new repetitions // in O(log n) allocations. var spacesByRepeatCount = ["", " "]; return function repeatSpaces(n) { var result = spacesByRepeatCount[n]; if (n && !result) { result = repeatSpaces(n >> 1); result += result; if (n & 1) { result += " "; } spacesByRepeatCount[n] = result; } return result; }; })(); /** * POD class to be manipulated by indentation fixers * @param {String} [rawText] original line text including * leading and trailing white space as tabs & spaces, but * NO line breaks. * @param {String} [text] content text with no leading or * trailing white space * @param {Number} [originalSpaces] indentation level with * tabs counted as multiple spaces. * @constructor */ function Line(rawText, text, originalSpaces) { this.rawText = rawText || ""; this.text = text || ""; this.originalSpaces = originalSpaces || 0; this.spaces = originalSpaces || 0; } function splitLines(rawText) { var oneTabWorthOfSpaces = repeatSpaces(config.spacesPerTab); return rawText.split(/\r\n|\r|\n/g).map(function (lineTxt) { lineRegex.lastIndex = 0; aligningTabRegex.lastIndex = 0; var sections = lineRegex.exec(lineTxt), rawLeadingWs = sections[1] || "", contentText = sections[2] || "", importantTabs = rawLeadingWs.replace( oneTabWorthOfSpaces, "\t" ), tabsAndAligningSpaces = aligningTabRegex.exec(importantTabs), // I don't like this as much as Lint does. importantWsLength = (tabsAndAligningSpaces[1] || "").replace(/[^\t]/g, "") .length * spacesPerTab + (tabsAndAligningSpaces[2] || "").length; return new Line(lineTxt, contentText, importantWsLength); }); } /** * Splits a string at occurrences of any of an array of separators. * @param {String} text * @param {Array<String|RegExp>} separators * @return {Array<String>} */ function splitOnArray(text, separators) { var result = [text]; separators.forEach(function (sep) { result = adp.concat.apply([], result.map( function (segment) { return segment.split(sep); } )); }); return result; } // for now, don't enforce token parity, tag parity, etc. etc. // (if I did, I would clearly want a grammar rather than a // regex-based system) // BUT I am abstracting the analyzer to take a paired token // input in case demand arises for that feature. /** * Class that encapsulates indentation rules based on paired * or regular tokens. The tokenSet parameter may be an object * {open: Array<String|RegExp> | String | RegExp, * close: Array<String|RegExp> | String | RegExp} * or an array of the same structures where open defines * tokens that demand an indent and close defines tokens that * demand a dedent. * @param {Object|Array<Object>} tokenSet * @constructor */ function IndentationAnalyzer(tokenSet) { var normalizedTokenSet = this.normalizedTokenSet = { open: [], close: [] }; (function normalize(defn) { var open, close; if (defn instanceof Array) { defn.forEach(normalize); } else { open = defn.open; close = defn.close; if (open) { if (open instanceof Array) { normalizedTokenSet.open = normalizedTokenSet.open.concat(open); } else if (open) { normalizedTokenSet.open.push(open); } } if (close) { if (close instanceof Array) { normalizedTokenSet.close = normalizedTokenSet.close.concat(close); } else { normalizedTokenSet.close.push(close); } } } })(tokenSet); } /** * Returns an analysis of the given string's indentation * properties: indent (total opening tokens present that * should affect the indentation of the next line), dedent * (total closing tokens that should affect indentation of * the next line), and prededent (closing tokens present that * should affect the indentation of the current line) * @param {String} text * @return {Object} */ IndentationAnalyzer.prototype.analyze = function (text) { var result = { prededent: 0, indent: 0, dedent: 0 }, openings, closings; openings = splitOnArray(text, this.normalizedTokenSet.open); result.indent = openings.length - 1; // now split those segments by the closing tokens // similarly, but keep the opening segments split up // so that pre-dedent can be calculated. closings = openings.map(function (segment) { return splitOnArray( segment, this.normalizedTokenSet.close ); }, this); // Each element of each array beyond one element is // due to 1 closing token result.dedent = closings.reduce(function (n, closing) { return n + closing.length - 1; }, 0); // only closing tokens that don't correspond to an opening // token on the same line affect the indentation of that // line. state is {opens, prededent} where opens is the // number of unclosed open tokens (each closing array is 1) // and prededent is the number of closing tokens without // a corresponding opening token. result.prededent = closings.reduce(function (state, closing) { var closedCount = closing.length - 1; if (closedCount > state.opens) { closedCount -= state.opens; state.opens = 0; state.prededent += closedCount; } else { state.opens -= closedCount; } state.opens++; return state; }, {prededent: 0, opens: 0}).prededent; return result; }; /** * Analyzes an array of Line objects assigning each of them * a 'spaces' property based on the balance of opening and * closing tokens found on each line and previous lines. * Mutates the Line objects - returns nothing. * @param {Array<Line>} lines */ IndentationAnalyzer.prototype.fix = function (lines) { var levelStack = [], level = 0, me = this; // Extend the line objects with indentation data from // analyzer. lines.forEach(function (line) { var analysis = me.analyze(line.text), pre = level - analysis.prededent, post = level + analysis.indent - analysis.dedent, parentDepth = levelStack.length; // Find the indentation of the level corresponding // to the last unopened closed token - that's this // line's indent. while (pre < level && parentDepth--) { level = levelStack.pop(); } line.spaces = levelStack.length * config.spacesPerTab; // If there was a net reduction in level, return to // the indentation that had the new level for the // next line. while (levelStack.length && level > post) { level = levelStack.pop(); } // If there wasn't an exact match searching down // or there was a net increase in level, push the old // level on the stack and let the new level be post. if (post !== level) { levelStack.push(level); level = post; } }); }; // populate the predefined language list with some common ones // that share a set of indentation rules // not sure where this will end up, so I use self-invoking lambda // to keep it cut-pasta-safe in case it leaves this scope. (function setupPredefinedLanguages(langs) { var htmlXml, curly, lisp; htmlXml = [ { // comments come first so the opening tag // doesn't match them open: /<!--/g, close: /-->/g }, { // <open tag> but not <self closing tag/> open: /<(?:"(?:\\"|[^"])*"|'(?:\\'|[^'])*'|[^>])*[^\/]>/g, // </closing tag> close: /<\/[^>]*>/g }, { // single tag split among lines open: "<", close: ">" } ]; // generic curly bracket language curly = [ {open: "{", close: "}"}, {open: "(", close: ")"}, {open: "[", close: "]"} ]; // curly would work for lisp, but performance is O(NM), // and lisp just needs M = 1. lisp = [{open: "(", close: ")"}]; langs.html = htmlXml; langs.xml = htmlXml; langs.c = curly; langs.cpp = curly; langs.cplusplus = curly; langs.js = curly; langs.javascript = curly; langs.ecmascript = curly; langs.java = curly; langs.lisp = lisp; })(predefinedLanguages); function renderLinesToString(lines) { return lines.map(function (line) { return repeatSpaces(line.spaces) + line.text; }).join("\r\n"); } function escapeForPreCode(text) { // as long as all of the '<' are escaped, the '>' seem to be fine. return text.replace("&", "&amp;").replace("<", "&lt;"); } function noTagCodeBlock(text) { return text.split(/\r\n|\r|\n/g).map(function (line) { return " " + line; }).join("\r\n"); } MicroToolbar = (function () { var templ =("<div>").addClass("fix-indent-toolbar"),
noMoBtn =
$("<button>").appendTo(templ) .text("X").addClass("fix-indent-no-more"), doItBtn =$("<button>").appendTo(templ)
langSel =
$("<select><option>language</option></select>") .appendTo(templ) .addClass("fix-indent-language"), sptLbl =$("<label>").appendTo(templ)
.text("spaces per tab"),
sptIn =
$("<input>").appendTo(templ) .attr("type", "number") .attr("min", "1") .attr("step", "1") .addClass("fix-indent-spaces-per-tab"), preCodeChk =$("<input>").appendTo(templ)
.attr("type", "checkbox")
preCodeLbl =
$("<label>").appendTo(templ) .text("use <pre><code>"); Object.keys(predefinedLanguages).forEach(function (key) {$("<option>").text(key).attr("value", key)
.appendTo(langSel);
});

function getUid(baseId) {
baseId = "fix-indent-" + baseId;
while ($("#" + baseId + "-unique-id").length) { baseId += "-very"; } return baseId + "-unique-id"; } function uniqueIds() { // can't be too careful with document scope IDs var sptId = getUid("spaces-per-tab"), preCodeId = getUid("use-pre-code"); sptIn.attr("id", sptId); sptLbl.attr("for", sptId); preCodeChk.attr("id", preCodeId); preCodeLbl.attr("for", preCodeId); } // Delegating events to the document element means that // pretty much everything in this module will be kept alive // until unload. In case it's running in an elevator // controller =D, remove the event handlers that have // references to this closure scope and let GC do its thing // when no more toolbars remain. function helpGc() { if (MicroToolbar.all().toolbarElement().length === 0) {$(document).off(".fixIndent");
}
}

// If the bookmarklet is run a second time, no need to re-hook
// all the events.
if ($(".fix-indent-toolbar").length === 0) {$(document)
.on("click.fixIndent",
".fix-indent-toolbar .fix-indent-no-more",
function () {
new MicroToolbar(this).die();
}
).on("change.fixIndent",
".fix-indent-toolbar .fix-indent-spaces-per-tab",
function () {
var my = new MicroToolbar(this);
my.spacesPerTab(my.spacesElement().val());
}
).on("change.fixIndent",
".fix-indent-toolbar .fix-indent-language",
function () {
var my = new MicroToolbar(this);
my.language(my.languageElement().val());
}
).on("change.fixIndent",
".fix-indent-toolbar .fix-indent-use-pre-code",
function () {
var my = new MicroToolbar(this);
my.usePreCode(my.preCodeElement().prop("checked"));
}
).on("click.fixIndent",
".fix-indent-toolbar .fix-indent-do-it",
function () {
var my = new MicroToolbar(this),
textEl = my.textElement().get(0),
selStart = textEl.selectionStart || 0,
selEnd = textEl.selectionEnd,
selText = textEl.value,
lines,
langObj = predefinedLanguages[config.language],
out;

if (!selEnd || selStart === selEnd) {
selEnd = textEl.textLength;
}
spacesPerTab = config.spacesPerTab;
selText = selText.slice(selStart, selEnd);
// pre & code tags could be part of the selection
if (/^<pre><code>/.test(selText)) {
selText = selText.slice(11);
}
if (/<\/pre><\/code>$/.test(selText)) { selText = selText.slice(0, -13); } lines = splitLines(selText); new IndentationAnalyzer(langObj).fix(lines); out = renderLinesToString(lines); if (config.usePreCode) { out = "<pre><code>\r\n" + escapeForPreCode(out) + "\r\n</pre></code>"; } else { out = noTagCodeBlock(out); } if (textEl.setRangeText) { textEl.setRangeText(out, selStart, selEnd); } else { textEl.value = textEl.value.slice(0, selStart) + out + textEl.value.slice(selEnd); } } ); } function MicroToolbar(el) { if (!(el instanceof$)) {
el = $(el || "body"); } if (!(this instanceof MicroToolbar)) { // used with no new gets an object referencing all toolbars // already constructed (it doesn't make any new toolbars) // (lint likes MicroToolbar.all() better, though) return new MicroToolbar(el.find(".fix-indent-toolbar")); } // it's fine to have an object referencing multiple toolbars // (since the config object is shared between all of them) // but there will be confusion if el is an ancestor with multiple // textareas some of which have the html toolbar and others don't. // So I have to narrow the query's selection, install a toolbar // on the textareas that need it, and make sure the element // referenced by the object has all of the toolbars, new & used. el = el.closest(":has(textarea)").find("textarea"); var tb = el.next(".fix-indent-toolbar"); if (tb.length !== el.length) { // update the template to reflect config before it's cloned sptIn.val(config.spacesPerTab); preCodeChk.prop("checked", config.usePreCode); el.filter(function () { return$(this).next(".fix-indent-toolbar").length === 0;
}).each(function () {
var oneEl = $(this), oneTb; uniqueIds(); oneTb = templ.clone(); oneEl.after(oneTb); tb.add(oneTb); }); } this.toolbarElement = function () { return tb; }; } MicroToolbar.prototype.spacesElement = function () { return this.toolbarElement().find(".fix-indent-spaces-per-tab"); }; MicroToolbar.prototype.languageElement = function () { return this.toolbarElement().find(".fix-indent-language"); }; MicroToolbar.prototype.preCodeElement = function () { return this.toolbarElement().find(".fix-indent-use-pre-code"); }; MicroToolbar.prototype.textElement = function () { return this.toolbarElement().prev("textarea"); }; MicroToolbar.prototype.spacesPerTab = function (n) { if (n !== undefined) { if (typeof n === "string") n = parseInt(n, 10); config.spacesPerTab = n; MicroToolbar.all().spacesElement().val(n); return this; } else { return config.spacesPerTab; } }; MicroToolbar.prototype.language = function (s) { if (s !== undefined) { if (predefinedLanguages.hasOwnProperty(s)) { config.language = s; MicroToolbar.all().languageElement().val(s); } return this; } else { return config.language; } }; MicroToolbar.prototype.usePreCode = function (b) { if (b !== undefined) { config.usePreCode = b; MicroToolbar.all().preCodeElement().prop("checked", b); return this; } else { return config.usePreCode; } }; MicroToolbar.prototype.die = function () { this.toolbarElement().remove(); helpGc(); }; MicroToolbar.all = function (inEl) { inEl = inEl instanceof$ ? inEl : $(inEl || document); return new MicroToolbar(inEl.find(".fix-indent-toolbar")); }; return MicroToolbar; })(); new MicroToolbar($("textarea"));

})(jQuery, {language: "java", spacesPerTab: 4, usePreCode: true});


Revision 14 adds the option to not do any language analysis and just prepare the selection as a markdown code block. This encompasses the "tabs to spaces only" mode that was originally planned and requested. The language selection control was split into a <select> and <label> like the other inputs. The default selection (previously "language" to indicate the purpose of the select box) was renamed to reflect its purpose. The list of predefined languages was prepended with generic language descriptions for the existing options.

There were changes in the IIFE setupPredefinedLanguages and in the IIFE whose result is MicroToolbar, but adding them here puts this post over the length limit, so see the fiddle for details.

I am now also specifically looking for feedback of the form "X feature is useless!" so that I can shorten the code.

• Hmm, your code is 20K+ characters, IE9 will only accept up to 5K. Jan 30 '14 at 15:41
• @konijn minification will cut all the spaces and line breaks, should be able to take all the variable and function names down to a single character, might make it down to 5k. Will have to try it & get back to you. Jan 30 '14 at 21:06
• @konijn got it down to -8k- 5.8k. will try fiddling with options, then start cutting stuff. Jan 30 '14 at 22:25
• In order to be rid of that last 800 characters, I'll have to shorten a bunch of property names, or add annotations for closure compiler and let it mangle them. Will look into it. Jan 30 '14 at 22:42
• Perhaps you will have to Golf it ;) Jan 30 '14 at 23:04

It was originally going to have a "just turn the tabs into spaces" mode, which is why you see it messing with tab/space alignment. Is this a feature that would be useful? It would take trivial effort to implement.

That feature might be useful on this site (Code Review):

• Reviewing the format of the code in the OP is on-topic; if the format is bad we should review that in an answer, not correct it in the OP's question.
• Occasionally, however, the formatting is messed up because the OP pasted-in code which contains tabs. In that case (formatting was messed up by the process of converting to markdown) it would be appropriate for someone else to edit the question to fix only that (tabs to spaces).
• This answer is community wiki because it really belongs on meta: it's not a code review. Feb 1 '14 at 23:22
• I absolutely agree that sensible indentation is an issue to address in the review if the code appears as the OP intended. I think it's mangled simply by copy/paste more often than it would first appear because @Jamal (bless him!) runs around tidying up new posts. The sample in the fiddle, for example, is an original post, verbatim, that he fixed manually, obtaining the same result that my tool does automatically. I was thinking simple tabs->spaces might be good for Python, too, but I don't really know. Feb 1 '14 at 23:37
• added the "markdown prep only" mode, which replaces tabs with spaces and adds 4 spaces or <pre><code>. Feb 4 '14 at 1:58
• Honestly, I think indentation should be fixed in an edit to encourage other reviewers. Leave a comment with the edit to this effect. But I digress. Feb 4 '14 at 4:01

The repeatSpaces function: I must have missed something.

Could you not use a built-in like String.repeat ? (works in Chrome 31)

If it is not available a shim like:
https://stackoverflow.com/a/202627/684890

String.prototype.repeat = function repeat(repeatTimes)
{
return new Array(repeatTimes+ 1).join(this);
};


would be sufficient?

• Doesn't look like it's in the standard, so I hesitate to use it regardless of adoption among common browsers. I think the shim you provide is about as straightforward as the way I've written it; if CR at large disagrees or if I test the performance and mine is inferior, I'll switch. Feb 4 '14 at 2:37
• @sqykly Thats an interesting idea. jsperf.com? Feb 4 '14 at 2:41
• Now I'm struggling to come up with a fair set of test cases since mine is using memoization; obviously I can't time generation of the same repeat count over and over, but I also can't use a set contrived to be devoid of repetitions. Thoughts? Feb 4 '14 at 3:16
• On my machine with memoization for both. It makes little difference. So I'd label my one as no real improvement. jsperf.com/code-review-40422 Feb 4 '14 at 3:36
• good call. I'm seeing 0 difference, too, except that firefox or my machine sucks =D. Feb 4 '14 at 4:09