4
\$\begingroup\$

I want to get a code review on the following transliteration code. I wrote it because there were some libraries that I have tried and they were specifically failing with the name "Yuliya" from Latin to Cyrillic. I wrote unit tests for my functions.

Things I noticed:

  • I don't need it to support both standards GOST 16876-71 and ISO 9-95, because I would use it to transliterate from/to Bulgarian and if you pay attention to CyrillicLowercase and LatinLowercase, it's a bit different than these standards.
  • Having CyrillicUppercase and CyrillicLowercase separately is not great. I believe this snippet solves it. I mean it doesn't have 4x const strings like I do, which is much better. It puts.ToLower() and .ToUpper() in the code.
  • you might ask why I did .OrderByDescending(x => x.Value.Length) specifically for LatinToCyrillic and the reason is because if you transliterate the name "Yuliya Koleva Hadzhiivanova" (Latin to Cyrillic), it won't think of Yu as a whole, instead it will think of it as Y and u, which would result in "Иулия..." instead of "Юлия...", which is wrong. I found that solution in this code and used it.
  • I used .Aggregate and this code uses .Replace. Which one is more optimal?
public sealed class Transliteration
{
    private const string CyrillicLowercase = "а б в г д е ж з и й к л м н о п р с т у ф х ц ч ш щ ъ ь ю я";
    private const string CyrillicUppercase = "А Б В Г Д Е Ж З И Й К Л М Н О П Р С Т У Ф Х Ц Ч Ш Щ Ъ Ь Ю Я";
    private const string LatinLowercase = "a b v g d e zh z i y k l m n o p r s t u f h ts ch sh sht a y yu ya";
    private const string LatinUppercase = "A B V G D E Zh Z I Y K L M N O P R S T U F H Ts Ch Sh Sht A Y Yu Ya";
    
    private readonly Dictionary<string,string> _dict;

    private Transliteration()
    {
        _dict = new Dictionary<string, string>();
    }

    private Transliteration(Dictionary<string, string> dict)
    {
        ArgumentNullException.ThrowIfNull(dict);
        
        _dict = dict;
    }
    
    public static Transliteration Create()
    {
        var lowercaseDict = CyrillicLowercase.Split(' ')
            .Zip(LatinLowercase.Split(' '), (k, v) => new { k, v })
            .ToDictionary(x => x.k, x => x.v);

        var uppercaseDict = CyrillicUppercase.Split(' ')
            .Zip(LatinUppercase.Split(' '), (k, v) => new { k, v })
            .ToDictionary(x => x.k, x => x.v);

        var both = lowercaseDict.Concat(uppercaseDict)
            .ToLookup(x => x.Key, x => x.Value)
            .ToDictionary(x => x.Key, g => g.First());

        return new Transliteration(both);
    }
    
    /// <summary>
    /// Transliterates cyrillic text to latin.
    /// </summary>
    /// <param name="text">The text to be transliterated.</param>
    /// <returns>The transliterated text.</returns>
    public string CyrillicToLatin(string text)
    {
        return _dict.Aggregate(text, (current, pair) => current.Replace(pair.Key, pair.Value));
    }
    
    /// <summary>
    /// Transliterates latin text to cyrillic.
    /// </summary>
    /// <param name="text">The text to be transliterated.</param>
    /// <returns>The transliterated text.</returns>
    public string LatinToCyrillic(string text)
    {
        return _dict.OrderByDescending(x => x.Value.Length).Aggregate(text, (current, pair) => current.Replace(pair.Value, pair.Key));
    }
}

xUnit

public class TransliterationTests
{
    private readonly Transliteration _transliteration;
    
    public TransliterationTests()
    {
        _transliteration = Transliteration.Create();
    }
    
    [Fact]
    public void CyrillicToLatinTextShouldEqualTheirEqual()
    {
        // Arrange

        
        // Act
        var actual = _transliteration.CyrillicToLatin("Юлия Колева Хаджииванова");

        // Assert
        actual.Should().Be("Yuliya Koleva Hadzhiivanova");
    }
    
    [Fact]
    public void LatinToCyrillicTextShouldEqualTheirEqual()
    {
        // Arrange

        
        // Act
        var actual = _transliteration.LatinToCyrillic("Yuliya Koleva Hadzhiivanova");

        // Assert
        actual.Should().Be("Юлия Колева Хаджииванова");
    }
}
\$\endgroup\$

1 Answer 1

6
\$\begingroup\$

Dynamic vs Static lookup table

  • You are recreating each and every time the same lookup table whenever someone calls the Create method
    • This is unnecessary since you can do it only once and without using linq
new Dictionary<char, string>
{
    { 'а', "a"}, { 'А', "A"},
    { 'б', "b"}, { 'Б', "B"},
    { 'в', "v"}, { 'В', "V"},
    { 'г', "g"}, { 'Г', "G"},
    { 'д', "d"}, { 'Д', "D"},
    { 'е', "e"}, { 'Е', "E"},
    { 'ж', "zh"}, { 'Ж', "Zh"},
    { 'з', "z"}, { 'З', "Z"},
    { 'и', "i"}, { 'И', "I"},
    { 'й', "y"}, { 'Й', "Y"},
    { 'к', "k"}, { 'К', "K"},
    { 'л', "l"}, { 'Л', "L"},
    { 'м', "m"}, { 'М', "M"},
    { 'н', "n"}, { 'Н', "N"},
    { 'о', "o"}, { 'О', "O"},
    { 'п', "p"}, { 'П', "P"},
    { 'р', "r"}, { 'Р', "R"},
    { 'с', "s"}, { 'С', "S"},
    { 'т', "t"}, { 'Т', "T"},
    { 'у', "u"}, { 'У', "U"},
    { 'ф', "f"}, { 'Ф', "F"},
    { 'х', "h"}, { 'Х', "H"},
    { 'ц', "ts"}, { 'Ц', "Ts"},
    { 'ч', "ch"}, { 'Ч', "Ch"},
    { 'ш', "sh"}, { 'Ш', "Sh"},
    { 'щ', "sht"}, { 'Щ', "Sht"},
    { 'ъ', "a"}, { 'Ъ', "A"},
    { 'ь', "y"}, { 'Ь', "Y"},
    { 'ю', "yu"}, { 'Ю', "Yu"},
    { 'я', "ya"}, { 'Я', "Ya"},
    { ' ', " " }
}
  • I've also added one extra mapping (space -> space)

Immutability

  • In your version you have used a const string which is immutable
  • In my version if I declare a field like this static readonly Dictionary<char, string> then the reference is immutable but the collection itself is mutable
    • In order to make the collection immutable we can use the ToImmutableDictionary builder method
private static readonly ImmutableDictionary<char, string> cyrillicToLatinMapping = new Dictionary<char, string>
{
    { 'а', "a"}, { 'А', "A"},
    { 'б', "b"}, { 'Б', "B"},
    { 'в', "v"}, { 'В', "V"},
    { 'г', "g"}, { 'Г', "G"},
    { 'д', "d"}, { 'Д', "D"},
    { 'е', "e"}, { 'Е', "E"},
    { 'ж', "zh"}, { 'Ж', "Zh"},
    { 'з', "z"}, { 'З', "Z"},
    { 'и', "i"}, { 'И', "I"},
    { 'й', "y"}, { 'Й', "Y"},
    { 'к', "k"}, { 'К', "K"},
    { 'л', "l"}, { 'Л', "L"},
    { 'м', "m"}, { 'М', "M"},
    { 'н', "n"}, { 'Н', "N"},
    { 'о', "o"}, { 'О', "O"},
    { 'п', "p"}, { 'П', "P"},
    { 'р', "r"}, { 'Р', "R"},
    { 'с', "s"}, { 'С', "S"},
    { 'т', "t"}, { 'Т', "T"},
    { 'у', "u"}, { 'У', "U"},
    { 'ф', "f"}, { 'Ф', "F"},
    { 'х', "h"}, { 'Х', "H"},
    { 'ц', "ts"}, { 'Ц', "Ts"},
    { 'ч', "ch"}, { 'Ч', "Ch"},
    { 'ш', "sh"}, { 'Ш', "Sh"},
    { 'щ', "sht"}, { 'Щ', "Sht"},
    { 'ъ', "a"}, { 'Ъ', "A"},
    { 'ь', "y"}, { 'Ь', "Y"},
    { 'ю', "yu"}, { 'Ю', "Yu"},
    { 'я', "ya"}, { 'Я', "Ya"},
    { ' ', " " }
}.ToImmutableDictionary();

Cyrillic 2 Latin

  • With this dictionary in our hand the translation is quite an easy task
string.Join("", text.ToCharArray().Select(c => cyrillicToLatinMapping[c])
  • First convert the text to a character array
  • Then loop through the array and do a lookup in a lookup table
  • Finally concatenate the looked up strings

Order by latin mappings' length

  • Yet again rather than doing this (re)ordering every time when someone calls the LatinToCyrillic method you can do it only once
private static readonly ImmutableArray<(string Latin, char Cyrillic)> latinToCyrillicMapping =
    cyrillicToLatinMapping
        .OrderByDescending(v => v.Value.Length)
        .Select(d => (Latin: d.Value, Cyrillic: d.Key))
        .ToImmutableArray();
  • Here we can't simple swap the Key and Value and still use a Dictionary to store the result (.ToImmutableDictionary(d => d.Value, d => d.Key)
  • Rather we could create an array with named ValueTuples

Latin 2 Cyrillic

  • With the above array in our hand the translation can be done like this
int startIdx = 0;
StringBuilder accumulator = new();
while (startIdx != text.Length)
{
    foreach (var (latin, cyrillic) in latinToCyrillicMapping)
    {
        if (text[startIdx..].StartsWith(latin))
        {
            accumulator.Append(cyrillic);
            startIdx += latin.Length;
            break;
        }
    }
}
return accumulator.ToString();
  • Here iterate through the text by moving a pointer
    • The pointer points to the next character which has not been translated yet
  • Since we have ordered latinToCyrillicMapping that's why we try to match (StartsWith) first the three letters long strings then the two letters long strings and finally the single letter long strings
  • Whenever we have found a match then we add the translation to the accumulator and move the pointer accordingly

For the sake of completeness here is the fully rewritten version of your Transliteration class

public static class Transliteration
{
    private static readonly ImmutableDictionary<char, string> cyrillicToLatinMapping = new Dictionary<char, string>
    {
        { 'а', "a"}, { 'А', "A"},
        { 'б', "b"}, { 'Б', "B"},
        { 'в', "v"}, { 'В', "V"},
        { 'г', "g"}, { 'Г', "G"},
        { 'д', "d"}, { 'Д', "D"},
        { 'е', "e"}, { 'Е', "E"},
        { 'ж', "zh"}, { 'Ж', "Zh"},
        { 'з', "z"}, { 'З', "Z"},
        { 'и', "i"}, { 'И', "I"},
        { 'й', "y"}, { 'Й', "Y"},
        { 'к', "k"}, { 'К', "K"},
        { 'л', "l"}, { 'Л', "L"},
        { 'м', "m"}, { 'М', "M"},
        { 'н', "n"}, { 'Н', "N"},
        { 'о', "o"}, { 'О', "O"},
        { 'п', "p"}, { 'П', "P"},
        { 'р', "r"}, { 'Р', "R"},
        { 'с', "s"}, { 'С', "S"},
        { 'т', "t"}, { 'Т', "T"},
        { 'у', "u"}, { 'У', "U"},
        { 'ф', "f"}, { 'Ф', "F"},
        { 'х', "h"}, { 'Х', "H"},
        { 'ц', "ts"}, { 'Ц', "Ts"},
        { 'ч', "ch"}, { 'Ч', "Ch"},
        { 'ш', "sh"}, { 'Ш', "Sh"},
        { 'щ', "sht"}, { 'Щ', "Sht"},
        { 'ъ', "a"}, { 'Ъ', "A"},
        { 'ь', "y"}, { 'Ь', "Y"},
        { 'ю', "yu"}, { 'Ю', "Yu"},
        { 'я', "ya"}, { 'Я', "Ya"},
        { ' ', " " }
    }.ToImmutableDictionary();

    private static readonly ImmutableArray<(string Latin, char Cyrillic)> latinToCyrillicMapping =
        cyrillicToLatinMapping
            .OrderByDescending(v => v.Value.Length)
            .Select(d => (Latin: d.Value, Cyrillic: d.Key))
            .ToImmutableArray();

    public static string CyrillicToLatin(string text)
        => string.Join("", text.ToCharArray().Select(c => cyrillicToLatinMapping[c]));

    public static string LatinToCyrillic(string text)
    {
        int startIdx = 0;
        StringBuilder accumulator = new();
        while (startIdx != text.Length)
        {
            foreach (var (latin, cyrillic) in latinToCyrillicMapping)
            {
                if (text[startIdx..].StartsWith(latin))
                {
                    accumulator.Append(cyrillic);
                    startIdx += latin.Length;
                    break;
                }
            }
        }
        return accumulator.ToString();
    }
}

UPDATE #1

In this case J is a char that cannot be found at LatinToCyrillicMapping and it is running in an infinite loop. I want to throw an exception instead of running in an infinite loop

var prevStartIdx = startIdx;
foreach (var (latin, cyrillic) in latinToCyrillicMapping)
{
    if (text[startIdx..].StartsWith(latin))
    {
        accumulator.Append(cyrillic);
        startIdx += latin.Length;
        break;
    }
}
if (prevStartIdx == startIdx)
    throw new NotSupportedException("...");
  • So, here we basically check whether or not the startIdx has advanced if not then throw an exception

UPDATE #2

text[startIdx..] allocates substring for each letter.
Use text.AsSpan() and then Slice() it

The modified code:

var textSpan = text.AsSpan();
...
var prevStartIdx = startIdx;
foreach (var (latin, cyrillic) in latinToCyrillicMapping)
{
    if (textSpan.Slice(startIdx).StartsWith(latin))
    {
        accumulator.Append(cyrillic);
        startIdx += latin.Length;
        break;
    }
}
if (prevStartIdx == startIdx)
    throw new NotSupportedException("...");
\$\endgroup\$
10
  • 1
    \$\begingroup\$ @nop Good catch :) I've tested my refactored version with your tests, I was not thinking about missing letters/special characters. \$\endgroup\$ Sep 28, 2022 at 17:20
  • 1
    \$\begingroup\$ @nop Okay, give me a couple of minutes and I'll update my answer. \$\endgroup\$ Oct 10, 2022 at 7:32
  • 1
    \$\begingroup\$ @nop Please check my update \$\endgroup\$ Oct 10, 2022 at 8:11
  • 1
    \$\begingroup\$ Thank you very much! \$\endgroup\$
    – nop
    Oct 10, 2022 at 8:29
  • 1
    \$\begingroup\$ text[startIdx..] allocates substring for each letter. Use text.AsSpan() and then Slice() it \$\endgroup\$ Dec 26, 2022 at 1:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.