1
\$\begingroup\$

I am attempting to polyfill the broken character encoding system in javascript. For example, characters such as 😒, have a length of 2 and are treated as 2 separate characters in regular expressions when they should have a length of 1 and be treated as a single character in regular expressions. This may not seem important, but I don't want to have a programming nightmare when I deploy my site multilingually with asian and african languages. I am attempting to polyfill the whole thing, but since I can't find any articles, this is the best I can do. I am attempting to detect the encoding based on the size of different characters since that is what I am, in essence, doing: polyfilling the size of the characters.

<script>
	// Charset Polyfiller
	let   charsettransfer = Object.create(null),
        maxUniChar = String.fromCodePoint(0x10ffff);
  let   needsPolyfill = true;
  
  if ('\xE2'.length === '\xE1'.length*2){
    charsettransfer.to = 'GB 18030';
    if ('\xE1'.length === 2){
      charsettransfer.from = 'ascii';
    } else if ('\xE1'.length === 1) {
      charsettransfer.from = 'UCS-16';
    }
  } else if ('\x7f'.length === 1 && '\x80'.length === 2){
    charsettransfer.from = 'ascii';
    charsettransfer.to = 'utf-7';
  } else if (maxUniChar.length === 4){
    charsettransfer.from = 'ascii';
    if ('\0'.length <= 1){
      charsettransfer.to = 'utf-8';
    } else if ('\u9999'.length === 3 && '\0'.length === 3){
      charsettransfer.to = 'utf-20';
    } else if ('\u9999'.length === 4) {
      charsettransfer.to = 'utf-32';
    } else {
      charsettransfer.to = 'utf-16';
    } 
  } else if (maxUniChar.length === 3) {
    charsettransfer.from = 'ascii';
    charsettransfer.to = 'utf-24';
		// utf-24 is my own nickname for when a browser uses only the minimal 
		// 24 bits for utf-32 (like python)
  } else if (maxUniChar.length === 2) {
    charsettransfer.from = 'ucs-16';
    charsettransfer.to = 'utf-16';
  } else if (maxUniChar.length === 1) {
    charsettransfer.from = 'ucs-32';
    charsettransfer.to = 'utf-32';
    needsPolyfill = false;
  }
  
	/*if ('\u9999'.length === 3){
		charsettransfer.from = 'ascii';
		charsettransfer.to = 'utf-8';
	} else if ('\0'.length === 4) {
		charsettransfer.to = 'utf-32';
    charsettransfer.from = 'ascii';
	} else if ('\0'.length === 3) {
		charsettransfer.to = 'utf-24';
		charsettransfer.from = 'ascii';
	} else if ('\0'.length === 2) {
		charsettransfer.to = 'utf-16';
		// utf-24 is my own nickname for when a browser uses only the minimal 
		// 24 bits for utf-32 (like python)
		charsettransfer.from = 'ascii';
	} else if ('\uD83D\uDE12'.length === 1){
		charsettransfer.from = 'ucs-32';
		charsettransfer.to = 'utf-32';
	}*/
  
  document.write('Your browser is attempting to render ' + charsettransfer.to
       + ' in ' + charsettransfer.from + '.\<br /> Your browser needing the polyfill is ' + needsPolyfill + '.');
</script>

How did I do? Am I on the right track? Are there any things I could add? Any improvements I could make? Any bugs that should be fixed? (sorry if i'm ranting, but i'm really concerned about this segment of code because LOTS of days of work are gonna depend on it, and I can't seem to find any helpful articles reguarding it).

\$\endgroup\$
1
  • \$\begingroup\$ Does it currently work as intended? \$\endgroup\$
    – Mast
    Commented Oct 22, 2017 at 14:10

1 Answer 1

3
\$\begingroup\$

Just don't. Standard practice is to use UTF-8 everywhere possible, and get rid of all other encodings. Especially if you have a multilingual system, dealing with multiple legacy encodings is a huge headache — just like this code above.

\$\endgroup\$
4
  • \$\begingroup\$ Yes, you are absolutely correct that I would prefer not to go through this headache. However, UTF-8 doesn't answer my question. Please read my entire question before posting a response. I am attempting to polyfill-ish (not an actual polyfill, rather just using my own custom functions) the size of characters so that anyone in any language can use my website utility project. I very much appreciate your enthusiasm, so it pains me to inform you that I will never accept this response because it doesn't answer any of my questions at all, and I would downvote your answer if I had enough reputation. \$\endgroup\$ Commented Jul 28, 2017 at 0:27
  • \$\begingroup\$ You did say polyfill, but still didn't give an explanation as to why you would want to support legacy encodings. \$\endgroup\$ Commented Jul 28, 2017 at 0:30
  • \$\begingroup\$ @3.1415926535897932384626433833 I think, while you don't like the answer, it's actually the most practical one. Serious organizations who have to deal with multilingual content, are going same exact way, (e.g. <meta charset="UTF-8"> <title>Wikipedia - Wikipedia</title>, <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> <title>World's largest online B2B marketplace-Alibaba.com</title>). Since you have not explained why you're solving the problem, it's fine to assume that avoiding the imaginary problem is an even better solution. Just my five cents. \$\endgroup\$ Commented Oct 19, 2017 at 5:44
  • \$\begingroup\$ i'm really concerned about this segment of code because LOTS of days of work are gonna depend on it -- why not depend on more reliable code than a home brewed solution? \$\endgroup\$ Commented Oct 19, 2017 at 5:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.