StringToHex functionality

[cristianbuse]

Hi Guido,

I would like to discuss performance of the method and I was not sure if you would like me to do a Pull Request. So, I will discuss here first and then you decide if you want me to do a PR or you want to push yourself.

Issues

Before touching on performance, I want to highlight a few things. Here is the current version of the method:

Public Function StringToHex(ByVal str As String) As String
    Dim i As Long
    Dim b() As Byte:      b = str
    Dim hexStr As String: hexStr = "0x" & Space(Len(str) * 4 + 2)

    For i = 1 To UBound(b) + 1
        Mid(hexStr, i * 2 + 1, 2) = Right$("0" & Hex$(b(i - 1)), 2)
    Next i
    StringToHex = hexStr
End Function

Firstly, I think we should allow for any type of string as input, not just UT16LE and so:

hexStr = "0x" & Space(Len(str) * 4 + 2)

should beccome:

hexStr = "0x" & Space(LenB(str) * 2)

BTW, the + 2 should not be there.

For example, something like StringToHex((StrConv("ABC", vbFromUnicode))) should return 0x414243 which can then be reversed with StrConv(HexToString("0x414243"), vbUnicode) which returns ABC.

---

Secondly, I would avoid the concatenation entirely, because it is slow. Obviously, the impact here is negligible as we're only concatenating once but we can still replace:

hexStr = "0x" & Space$(LenB(str) * 2)

with:

hexStr = Space$(LenB(str) * 2 + 2)
Mid$(hexStr, 1, 2) = "0x"

Performance

We could use a mapping to avoid creating strings in a loop for each character. Something like:

Public Function StringToHex(ByVal s As String) As String
    Dim i As Long
    Dim b() As Byte:      b = s
    Dim hexStr As String: hexStr = Space$(LenB(s) * 2 + 2)
    Static map(0 To 255) As String
    Static isInitialized As Boolean
    '
    If Not isInitialized Then
        For i = 0 To 255
            map(i) = Right$("0" & Hex$(i), 2)
        Next i
        isInitialized = True
    End If
    '
    Mid$(hexStr, 1, 2) = "0x"
    For i = LBound(b) To UBound(b)
        Mid$(hexStr, (i + 1) * 2 + 1, 2) = map(b(i))
    Next i
    StringToHex = hexStr
End Function

This version of the function is about 6 times faster than the original.

---

There is another approach, but it turns out it is slower than the one presented above, but I will put it here just for reference:

Public Function StringToHex(ByVal s As String) As String
    Dim i As Long
    Dim j As Long
    Dim b() As Byte:      b = s
    Dim hexB() As Byte:   ReDim hexB(-3 To LenB(s) * 4)
    Dim hexS As String
    Dim currB As Byte
    Static mapHi(0 To 255) As Byte
    Static mapLo(0 To 255) As Byte
    Static isInitialized As Boolean
    '
    If Not isInitialized Then
        For i = 0 To 255
            hexS = Right$("0" & Hex$(i), 2)
            mapHi(i) = Asc(Left$(hexS, 1))
            mapLo(i) = Asc(Right$(hexS, 1))
        Next i
        isInitialized = True
    End If
    '
    hexB(-3) = 48 'Asc("0")
    hexB(-1) = 120 'Asc("x")
    For i = LBound(b) To UBound(b)
        j = i * 4 + 1
        hexB(j) = mapHi(b(i))
        hexB(j + 2) = mapLo(b(i))
    Next i
    StringToHex = hexB
End Function

It's still about 4-5 times faster than the original but about 20% slower than the one using a string map.

---

Actually, there is one way to improve even further. If we minimize the number of times the result string is being copied then we get an extra 10-20% improvement:

Public Function StringToHex(ByVal s As String) As String
    Dim i As Long
    Dim b() As Byte:      b = s
    Dim hexStr As String: StringToHex = Space$(LenB(s) * 2 + 2)
    Static map(0 To 255) As String
    Static isInitialized As Boolean
    '
    If Not isInitialized Then
        For i = 0 To 255
            map(i) = Right$("0" & Hex$(i), 2)
        Next i
        isInitialized = True
    End If
    '
    Mid$(StringToHex, 1, 2) = "0x"
    For i = LBound(b) To UBound(b)
        Mid$(StringToHex, (i + 1) * 2 + 1, 2) = map(b(i))
    Next i
End Function

[guwidoe]

we can still replace:

hexStr = "0x" & Space$(LenB(str) * 2)

with:

hexStr = Space$(LenB(str) * 2 + 2)
Mid$(hexStr, 1, 2) = 

In my head, this was already how it was done, that's where the + 2 came from 😂 Thanks for pointing out this mistake!

Thanks for testing the mapping approach and pointing out that it is so much faster, that is good to know and please, go ahead with pull requests, of course, I'm open for such suggestions!

I think minimizing string copying as in your last suggestion is also a good practice.

Would you like to create a PR or should I fix it?

I'm wondering why you are concerned with the performance of this method, as I think it's already more than good enough for all the likely use-cases for it?

I'm more concerned with the performance of the ReplaceUnicodeLiterals function as it is of N * M complexity where N is the length of the string and M is the number of different Unicode literals in the string. I think this function should be rewritten to have linear time complexity (with string length), but I was too lazy to do so so far.

[cristianbuse]

I'm wondering why you are concerned with the performance of this method

Honestly, I'm not - I think this method is useful for testing and maybe initializations. But because I'm quite busy, I thought I'd start top-down and it was also easier to start with the smallest method. My goal is to review each and every method, because this is a great library and I want to help as much as I can 😄 plus I hope it becomes standard within the VBA community - I for sure will use it.

I will create a PR later today - it would be my first ever BTW and I will learn something new.

[guwidoe]

Awesome! I too hope it'll be useful for many people and that's why I'm currently working on a complete wrapper for the transcoding API functions for Mac and windows, similar to your gist but including all the possible encodings.

[cristianbuse]

Dealt with in 5c38d40