Your code looks pretty got but there are some things that you can simplify, probably the switch cases are the most unwanted part of your code.

private string _path;
public string RelativePath => _path;

You can make this into a property with get only

public string RelativePath { get; }

You can use expression bodies for all of your get only properties, without a backing field :

public override string AbsolutePath
{
    get
    {
        return $"{Scheme}://application:,,,{RelativePath}";
    }
}

Can become

public override string AbsolutePath => $"{Scheme}://siteoforigin:,,,{RelativePath}";

You can do the same for all of the AbsolutePath properties.

private string _assemblyShortName; can be a readonly variable

The cast here is redundant and so is the entire line.

public Uri ToUri() => this;

You already have an implicit operator that does that for you

public static implicit operator Uri(PackUriBuilder builder)

Speaking of which You can replace the nasty switch case with a Dictionary

public static implicit operator Uri(PackUriBuilder builder)
{
    switch (builder._uriKind)
    {
        case UriKind.Absolute:
            return new Uri(builder.ToString(), UriKind.Absolute);
        case UriKind.Relative:
             return new Uri(builder.RelativePath, UriKind.Relative);
    }
        // todo: I know this needs a better message ;-)
   throw new ArgumentOutOfRangeException("Invalid UriKind.");
}

With a dictionary looking similar to this one

protected static Dictionary<UriKind, Func<PackUriBuilder, Uri>> helper = new Dictionary
        <UriKind, Func<PackUriBuilder, Uri>>
        {
            {UriKind.Absolute, builder => new Uri(builder.ToString(), UriKind.Absolute)},
            {UriKind.Relative, builder => new Uri(builder.RelativePath, UriKind.Relative)}
        };

Can become

public static implicit operator Uri(PackUriBuilder builder) => helper[builder._uriKind].Invoke(builder);

Your public override string ToString() is similar but it might be a little bit more harder to do with a dictionary, since there are some virtual variables involved, namely AbsolutePath and since you can't make it into a static variable and initialize it like the helper dictionary used for the implicit operator, You might need to initialize the dictionary from the constructor but as you know virtual calls can be very tricky if called in the constructor.