Resolve license issue with the utf8proc code. #364
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update utf8proc.[ch] to use the version now maintained by the Julia Language project (https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md). The license for the previous version was unacceptable for the Debian and Ubuntu release systems. The new version both updates the code and addresses the license issue. It turns out that the utf8proc software we are using was turned over to the Julia Language developers and the license terms changed to allow modification. (https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md). So the fix here is as follows: 1. Wrap the library with a fixed interface: libdispatch/dutf8.c and include/ncutf8.h. 2. Replace the existing utf8proc code with the new version from https://github.com/JuliaLang/utf8proc. 3. Add a couple more test cases: nc_test/tst_utf8_validate.c and nc_test_utf8_phrases.c. If/when I can find a usable normalization test, I will incorporate that later.
|
There are some build failures on Windows with Visual Studio. I'll get them resolved and then merge this pull request. |
|
Sorry, I always forget to test with cmake.
=Dennis
…On 2/17/2017 1:04 PM, Ward Fisher wrote:
There are some build failures on Windows with Visual Studio. I'll get
them resolved and then merge this pull request.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#364 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AA3P29mw4KuXxcNOpjkM3pLggH8eMnWHks5rdf1fgaJpZM4MDkUB>.
|
|
Not a problem. The root of the issue is described here, and should be easy enough for me to fix. Attending to it now. |
|
For what its worth, I only use a couple of the functions in utf8proc. None of the others |
|
Propagating changes from |
DennisHeimbigner
added a commit
that referenced
this pull request
Jun 6, 2017
This is a follow-on in that the old utf8 code was still being used in ncgen to convert utf8->utf16 when converting cdl to Java (see genj.c). The new code apparently has no utf16 support, but it does have utf32 support. Converting utf32 -> utf16 can be approximated by truncating the 32bits to 16 bits, unless the top 16 bits are not zero. This latter condition is unlikely to be common because it implies use of some rather obscure characters. So solution is to convert to utf32 and truncate to 16 bits to get utf16. An error is reported if the high-order truncated 16 bits are not zero. If we get complaints, then I will figure out how to convert full utf32 to a utf16 pair. Also removed the old code from ncgen.
DennisHeimbigner
added a commit
that referenced
this pull request
Jun 19, 2017
This is a follow-on in that the old utf8 code was still being used in ncgen to convert utf8->utf16 when converting cdl to Java (see genj.c). The new code apparently has no utf16 support, but it does have utf32 support. Converting utf32 -> utf16 can be approximated by truncating the 32bits to 16 bits, unless the top 16 bits are not zero. This latter condition is unlikely to be common because it implies use of some rather obscure characters. So solution is to convert to utf32 and truncate to 16 bits to get utf16. An error is reported if the high-order truncated 16 bits are not zero. If we get complaints, then I will figure out how to convert full utf32 to a utf16 pair. Other changes: 1. removed the old code from ncgen. 2. changed UTF8PROC_DLLEXPORT (in utf8proc) to EXTERNL and added appropriate includes. This should fix issue #404, but since we cannot duplicate the failure, I am not quite sure.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Re: Github issue #349.
Update utf8proc.[ch] to use the version now
maintained by the Julia Language project
(https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md).
The license for the previous version was
unacceptable for the Debian and Ubuntu release
systems. The new version both updates the code
and addresses the license issue.
It turns out that the utf8proc software we are using
was turned over to the Julia Language developers
and the license terms changed to allow modification.
(https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md).
So the fix here is as follows:
and include/ncutf8.h.
from https://github.com/JuliaLang/utf8proc.
and nc_test_utf8_phrases.c. If/when I can find a usable
normalization test, I will incorporate that later.