I read before that dealing with UTF8 was non-trivial with Ada before but was never concerned, and honestly I only needed to write a puny rightward arrow… But pardon my French, holy ßðit !!!
So I only had to write three line using Wide_wide_String and Put from the related package. That’s all. The symbol was displayed, and only è wasn’t: Put_line from Wide_Wide_Text_IO ALWAYS mangles that accentuated character. Text_IO doesn’t.
I tried -gnatW8, I tried pragma Wide_Wide_Encoding(UTF8), nothing works.
See the code:
pragma Extensions_Allowed(All);
pragma Wide_Character_Encoding(UTF8);
with Ada.Text_IO; use Ada.Text_IO;
with Ada.Wide_Wide_Text_IO;
with Ada.Characters.Conversions;
use Ada.Characters.Conversions;
procedure genseq is
package TIO renames Ada.Wide_Wide_Text_IO;
subtype WWString is Wide_Wide_String;
subtype WWCharacter is Wide_Wide_Character;
function V(S: String) return String is ('[' & S & "](V)") with Inline;
function R(S: String) return String is ('[' & S & "](R)") with Inline;
function V(S: Character) return String is (V(String'(1=>S)));
function R(S: Character) return String is (R(String'(1=>S)));
function TWWS (S:String) return Wide_Wide_String renames Ada.Characters.Conversions.To_Wide_Wide_String;
function TWWC (S:Character) return Wide_Wide_Character renames Ada.Characters.Conversions.To_Wide_Wide_Character;
procedure NVVV (S:String) is
A renames S(1);
B renames S(2);
C renames S(3);
function AABC return String is (A&V(S)) with Inline;
function ABBC return String is (R(A)&B&V(B&C)) with Inline;
function ABCC return String is (V(A)&R(B)&C&V(C)) with Inline;
begin
TIO.Put("{{< ne >}}" & TWWS(String'(1=>A)) & "→" & TWWS(V(B)) & "→" & TWWS(R(C)));
for Ind in 1..3 loop
Put (" " & AABC);
Put (" " & ABBC);
Put (" " & ABCC);
end loop;
Put_Line (" " & V(S) & " " & V(S) & " " & V(A&B) & C & "{{</ ne >}}");
end NVVV;
procedure NNVR(S:String) is
A renames S(1);
B renames S(2);
C renames S(3);
function AABC return String is (A&A&V(B)&R(C)) with Inline;
function ABBC return String is (A&B&V(B)&R(C)) with Inline;
function ABCC return String is (A&R(B)&C&R(C)) with Inline;
begin
TIO.Put("{{< ne >}}" & TWWS(String'(1 => A)) & "→" & TWWS(V(B)) & "→" & TWWS(R(C)));
for Ind in 1..3 loop
Put (" " & AABC);
Put (" " & ABBC);
Put (" " & ABCC);
end loop;
Put_Line (" " & A & V(B) & R(C) & " " & A & V(B) & R(C) & " " & A & V(B) & C & "{{</ ne >}}");
end NNVR;
procedure NVRN(S:String) is
A renames S(1);
B renames S(2);
C renames S(3);
function AABC return String is (A&V(A)&R(B)&C) with Inline;
function ABBC return String is (R(A)&B&R(B)&C) with Inline;
function ABCC return String is (A&R(B)&C&R(C)) with Inline;
begin
TIO.put(TWWC(A) & "→" & TWWS(R(B)) & "→" & TWWC(C));
for Ind in 1..3 loop
Put (" " & AABC);
Put (" " & ABBC);
Put (" " & ABCC);
end loop;
for Ind in 1..3 loop
Put (" " & V(A) & R(B) & C);
end loop;
Put_line("{{</ ne >}}");
end NVRN;
procedure NRNV(S:String) is
A renames S(1);
B renames S(2);
C renames S(3);
function AABC return String is (A&R(A)&B&V(C)) with Inline;
function ABBC return String is (R(A)&B&B&V(C)) with Inline;
function ABCC return String is (R(A)&B&C&V(C)) with Inline;
begin
TIO.put("{{< ne >}}" & TWWS(R(A)) & "→" & TWWC(B) & "→" & TWWS(R(C)));
for Ind in 1..3 loop
Put (" " & AABC);
Put (" " & ABBC);
Put (" " & ABCC);
end loop;
Put_line (" " & R(A) & B & V(C) & " " & R(A) & B & V(C) & " " & R(A) & B & C & "{{</ ne >}}");
end NRNV;
procedure NNVV(S:String) is
A renames S(1);
B renames S(2);
C renames S(3);
function AABC return String is (A&A&V(B&C)) with Inline;
function ABBC return String is (A&R(B)&B&V(C)) with Inline;
function ABCC return String is (V(A&B)&R(C)&C) with Inline;
begin
TIO.put(TWWC(A) & "→" & TWWC(B) & "→" & TWWC(C));
for Ind in 1..3 loop
Put (" " & AABC);
Put (" " & ABBC);
Put (" " & ABCC);
end loop;
Put_line (" " & V(S) & " " & V(S) & " " & V(A&B) & A & "{{</ ne >}}");
end NNVV;
procedure Normal (S:String) is
A renames S(1);
B renames S(2);
C renames S(3);
function AABC return String is ('*'&A&'*'&A&B&C) with Inline;
function ABBC return String is ('*'&A&'*'&B&B&C) with Inline;
function ABCC return String is ('*'&A&'*'&B&C&C) with Inline;
begin
TIO.Put("{{< ne >}}" & TWWC(A) & "→" & TWWC(B) & "→" & TWWC(C));
for Ind in 1..3 loop
Put(" " & AABC & " " & ABBC & " " & ABCC);
end loop;
for Ind in 1..3 loop
Put(" " & '*' & A & '*' & B & C);
end loop;
Put_line("{{</ ne >}}");
end Normal;
WithNormal: Boolean := Boolean'Value(Get_Line);
begin
Skip_Line;
while not End_Of_File loop
declare
IncorrectDataLength: exception;
Choice: String := Get_Line;
Data: String := Get_Line;
begin
raise IncorrectDataLength when Data'Length not in 3..4;
if WithNormal then Normal(Data); end if;
case Positive'Value(Choice) is
when 1 => NVVV(Data);
when 2 => NNVR(Data);
when 3 => NVRN(Data);
when 4 => NRNV(Data);
when 5 => NNVV(Data);
when others => raise Constraint_Error;
end case;
exception
when Constraint_Error => Put_line(Current_Error, Choice & "line " & Line'Image & ": Invaid pattern"); exit;
when IncorrectDataLength => Put_line(Current_Error, Data & "line " & Line'Image & ": Invalid Data length (" & Data'Length'Image & ')'); exit;
when others => Put_line(Current_Error, "Lacks a pattern or data"); exit;
end;
end loop;
exception
when End_Error => Put_line(Current_Error, "A line is missing");
end genseq;
try with this (standard) input file:
False
1
mèb
my terminal is top-notch and it’s è we’re talking about, not some obscure complex unicode glyph.
I noticed something similar for something I was working on. Out of curiosity, try piping the output into cat and see if it still displays wrong? I found that my terminal couldn’t handle it natively, but some programs could display it properly. Make sure that you still have -gnatW8 enabled for this test.
UTF-8 with GNAT is very simple if you understand how it works. In essence the -gnatW8 switch corrupts all I/O subsystem and string stream attributes. In effect UTF-8 would be decoded into UCS-2/4 and output would be encoded into UTF-8. Note that corruption is done on the run-time level. It means that the -gnatW8 takes precedence over the code compiled before without it! This means that a tested library can suddenly behave in a completely different way!
Consider this sample program to understand how -gnatW8 works:
with Ada.Text_IO;
with Ada.Wide_Wide_Text_IO;
procedure Main is
E_With_Grave : constant := 16#E8#;
begin
Ada.Text_IO.Put_Line ("Text_IO " & Character'Val (E_With_Grave));
Ada.Wide_Wide_Text_IO.Put_Line ("Wide_Wide_Text_IO " & Wide_Wide_Character'Val (E_With_Grave));
end Main;
If you build it without -gnatW8. It will output Latin-1 as specified and you will see this
Text_IO ▒
Wide_Wide_Text_IO ▒
on a UTF-8 console. I.e. it will not touch anything. If you build with -gnatW8, both will be encoded to UTF-8:
Text_IO è
Wide_Wide_Text_IO è
My recommendation is even simpler:
Never use any Wide strings and I/O;
Never use -gnatW8. Make sure that your compiler does not have it as the default;
Assume String UTF-8 encoded;
As a consequence you will lose Unicode literals and identifiers, which is no loss at all IMO.
… I was wrong again.
There is this variable S, in the code above.
Outputting it as whole is fine: Put(S). Put(1..3) is fine too.
But Put((S(1)) or Put(S(2)), it’s mangled.
What kind of bug is that ?!
And worse every time I change the code, I can’t single it out.
I’ve had enough of this… I hope someone compile the example and figure it out.
It is a bug by design. When reading Unicode into String you have a bad and worse choices. Let you attempt to decode it then beyond Latin-1 all input will suddenly become illegal and you will have to raise Data_Error which nobody will ever accept. So -gnatW8 encodes string output but not decodes it back.
No, -gnatW8 consistently corrupts both singular characters and strings. At least GNAT 13.3.0 does so.
Ok, is there a good Ada solution that doesn’t involve breaking the knee and using outside libraries ? Sorry to say this, but what have the developers/designers/I-don’t-give-a-damn been doing ? I can read and output most of Unicode in both my browsers, terminal, text edtior, etc, all written either in C, C++, Rust or Go. But Ada (or it’s environment, not my problem) does that ?
Those are rhetorical questions, Do not feel obliged to respond.
You should stay away from using anything but Latin_1 in Ada.
Time is wasted on dealing with the mess of standard library string types.
Don’t use anything with Wide_* in name, don’t even use (Un-)bounded_Strings in Ada.
If you feel the need to, either think harder and solve with String and Latin_1 or change the language until Ada manages to tackle this.
For real ? I drop Ada for months and the moment I find pleasure to use it practically on things I can handle, I find the one sore point ugly to warrant this advice ?
For a damn è ?!??
This is nuts.
Isn’t there a way to read è characters correctly and consistently ? I can avoid crazy emojis, but I’m French, putain de merde. I can’t output my own language now !
I also had a lot of problems with Adas String handling and it took me some time to get used to it. I’m still not a huge fan but I think that’s just because of the way strong typing is. It feels kinda wrong having to cast from Unbounded to Bounded and back and more.
You would and you should use a library that handles these strings for you. I haven’t used them but this resource should guide you more than I can do.
You’re being a bit dramatic. This is an old language.
I don’t know why some Latin_1 are not printable (they should).
In any case, Ada lacks a proper abstraction for the “string”.
Use ‘e’ until situation improves.
It’s because you shouldn’t do it.
It’s perfectly fine for an old language to have sections in the standard library that you should not touch.
And you shouldn’t touch Wide_* and (Un)bounded_Strings because they are an unergonomic mess.
It is very likely not Ada the language, but rather (a) GNAT mangling things, or (b) your terminal.
Assuming you’re using Windows, for the terminal:
Ensure you are using the correct codepage: chcp 65001.
Ensure the terminal is using a unicode font, right-click the title-bar or system-menu, click Properties, and select “Lucida Console” (I think “Consolas” works, too, but I might be misremembering.
You can verify correct transmission via GPS’s “Run” tab; this compensates for GNAT’s… odd runtime.
As for GNAT, I never use -gnatW8, at least not directly. (Using GPS.) Instead:
Right-click your source, select Properties, set Character Set to “Unicode UTF-8”.
Put Pragma Wide_Character_Encoding( UTF8 ); at the top of the source.
I disagree, the standard has since Ada 2012 defined Identifiers in terms of Unicode.
This is a GNAT problem, not an Ada problem.
If you find the situation intolerable, consider contributing to another Ada implementation than GNAT. There’s my Byron & BAATS compilers, the HAC compiler, and a few new ones. — The presence of a viable non-GNAT open source Ada will change how willing people are to jump in on GNAT… and it will help keep stupid stuff out of the language, like AdaCore’s disgusting proposal for a Class construct/syntax-sugar for tagged types.
I use voidlinux, terminal Kitty 0.45 and fr_FR.UTF-8 as a local. Also I tested with libreoffice too. Terminal’s innocent. I’ll participate to your project to my capacity.
