Change StringLiteral mapping for wide chars and add a test case.

[wilsonk]

I was getting some errors when trying to build a library with something similar to the test case. The mapping seems to work in one direction, but going back from D->C++ still gives an error. The type getting through to cpptypes.cpp(1819):toType is a 'const(int)*' and it appears as though that int is not being seen as a builtin type? The switch(t->ty) will match against a Tint32, which I figured should map easily to a builtin type with toClang[t] in the line above?? Not sure what is going on there.

Thanks,
Kelly

[Syniurge]

Hi Kelly,

String literals were correctly mapped I think, the error with wchar_t is probably in cpptypes.cpp. wchar_t doesn't have a D equivalent so it's being mapped to an integer type, but toType() should be sending back wchar_t, not int. Its t->ty is Tint32 but it's actually a cpp::TypeBasic which remembers the corresponding Clang type.

I'll investigate after I manage to make the Windows build work. Did you try to compile the STL examples back in February? I've also been trying to use the MingW-x64 standard lib/linker but it's far from straightforward (thanks to hardcoded paths in Clang..)

[Syniurge]

Woops I misread your post, const(int)* on the D side is correct, so the problem may be elsewhere.

[wilsonk]

Hello Elie,

If string literals were mapped correctly for wide chars then the example I provided would compile (without any use in D, just modmapping and importing), but it doesn’t compile…that is why I looked into this. I ran into this when trying to compile/use the Irrlicht3D library and then made the minimal example from there. When compiling the minimal example in showcase the call to SL->getString asserts…it has to be getBytes to compile the example, I think.

I am pretty sure that I only tried to compile showcase back in February. I REALLY wouldn’t bother with mingw-x64 right now, as it is a pain to interoperate with clang even under normal circumstances. If you can get things to run with MSVC, it will be a small miracle at this point…so that is probably good enough for now :)

Did you get the MS vtables to work yet?

Thanks,

Kelly

From: Elie Morisse [mailto:[email protected]]
Sent: Sunday, April 19, 2015 8:10 AM
To: Syniurge/Calypso
Cc: wilsonk
Subject: Re: [Calypso] Change StringLiteral mapping for wide chars and add a test case. (#11)

Hi Kelly,

String literals were correctly mapped I think, the error with wchar_t is probably in cpptypes.cpp. wchar_t doesn't have a D equivalent so it's being mapped to an integer type, but toType() should be sending back wchar_t, not int. Its t->ty is Tint32 but it's actually a cpp::TypeBasic which remembers the corresponding Clang type.

I'll investigate after I manage to make the Windows build work. Did you try to compile the STL examples back in February? I've also been trying to use the MingW-x64 standard lib/linker but it's far from straightforward (thanks to hardcoded paths in Clang..)

—
Reply to this email directly or view it on GitHub #11 (comment) .

[wilsonk]

Hello Elie,

Is modulemap’ing a C header supposed to expose everything in the header to the D files being compiled? So when using the stdc modulemap that exports everything in the ctype.h file, can we call all the functions in ctype.h? (And get access to all global vars, etc.)?

I can’t seem to access any funcs or vars from any C header files whether compiling one D file or multiple files…and I have tried modulemap’ing more than one .h file, etc. Since there is no namespace, I tried using “cpp.” as the namespace but that didn’t work either.

You mentioned on newsgroup that we should be able to use C/C++ libs, so I figured we would at least be able to access global funcs. Maybe I am just doing something wrong?

Thanks,

Kelly

From: Elie Morisse [mailto:[email protected]]
Sent: Sunday, April 19, 2015 8:26 AM
To: Syniurge/Calypso
Cc: wilsonk
Subject: Re: [Calypso] Change StringLiteral mapping for wide chars and add a test case. (#11)

Woops I misread your post, const(int)* on the D side is correct, so the problem may be elsewhere.

—
Reply to this email directly or view it on GitHub #11 (comment) .

[Syniurge]

Hi Kelly,

Currently to load a module map you have to specify it with -cM=stdc.modulemap (so there can only be one module map, but that option is temporary).

There's another hacky option, -cMH to specify the root directory where Calypso looks for headers:

-cM= - Clang module map file
-cMH= - Root directory to look for headers specified by Clang modules (default: /usr/include)

So it's still a hack, for proper support Calypso should detect the module map files during the PCH generation and e.g build a list in another calypso_cache.something file.

[Syniurge]

To load stdc.ctype:

modmap (C++) "ctype.h";
import (C++) cpp.stdc.ctype;

and yes it's supposed to import global functions, variables, and anonymous tags (for example the anonymous enum in ctype.h).

[wilsonk]

Hello Elie,

I knew about the -cM and -cMH options…they are in the help message, but it is good to know that only one module can be loaded at a time and that I need to specify the -cMH option if my header isn’t in /usr/include.

Loading the stdc.ctype modulemap actually only works here with ‘import (C++) stdc.ctype;’?!? Not sure if you are seeing the same thing.

Adding this modulemap to the showcase example allows things to compile with the proper -cM options, but I still can’t access any variables or functions. Have you actually used some ctype variables? Maybe the anonymous enum you spoke of? Or did you just get it to compile and not use any func/vars?

The reason I am doing this is that libzmq is pretty close to working right now but there is one header (and corresponding .cpp file) that contains a bunch of functions for starting up zmq sockets. I can rewrite them in D…but that is the point of Calypso, not to have to rewrite this stuff :)

I managed to get the modulemap for zmq up and compiling here…almost. I get this error when compiling a simple main function with the modulemap and the minimum “zmq.hpp” imported:

ldc2: /home/wilsonk/Downloads/llvm-3.6/llvm/tools/clang/include/clang/Basic/SourceLocation.h:134: clang::SourceLocation clang::SourceLocation::getLocWithOffset(int) const: Assertion `((getOffset()+Offset) & MacroIDBit) == 0 && "offset overflow"' failed.

#0 0x3a4ce7c llvm::sys::PrintStackTrace(_IO_FILE*) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x3a4ce7c)

#1 0x3a4d117 PrintStackTraceSignalHandler(void*) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x3a4d117)

#2 0x3a4bd39 SignalHandler(int) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x3a4bd39)

#3 0x7fb9c3e82340 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x10340)

#4 0x7fb9c2c7dcc9 gsignal /build/buildd/eglibc-2.19/signal/../nptl/sysdeps/unix/sysv/linux/raise.c:56:0

#5 0x7fb9c2c810d8 abort /build/buildd/eglibc-2.19/stdlib/abort.c:91:0

#6 0x7fb9c2c76b86 __assert_fail_base /build/buildd/eglibc-2.19/assert/assert.c:92:0

#7 0x7fb9c2c76c32 (/lib/x86_64-linux-gnu/libc.so.6+0x2fc32)

#8 0x142f2a9 clang::SourceLocation::getLocWithOffset(int) const (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x142f2a9)

#9 0x186e468 clang::ASTReader::FindFileRegionDecls(clang::FileID, unsigned int, unsigned int, llvm::SmallVectorImplclang::Decl*&) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x186e468)

#10 0x129b6e3 cpp::reclang::ASTUnit::findFileRegionDecls(clang::FileID, unsigned int, unsigned int, llvm::SmallVectorImplclang::Decl*&) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x129b6e3)

#11 0x1286a12 cpp::mapClangModule(cpp::DeclMapper&, clang::Module_, Array<Dsymbol_>*) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x1286a12)

#12 0x128704c cpp::Module::load(Loc, Array<Identifier*>, Identifier) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x128704c)

#13 0x13bf7b5 cpp::Import::loadModule(Loc, Array<Identifier*>, Identifier) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x13bf7b5)

#14 0x112adf4 Import::load(Scope*) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x112adf4)

#15 0x13bf6b9 cpp::Import::load(Scope*) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x13bf6b9)

#16 0x112af02 Import::importAll(Scope*) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x112af02)

#17 0x11eb7d0 Module::importAll(Scope*) (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x11eb7d0)

#18 0x1103452 main (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x1103452)

#19 0x7fb9c2c68ec5 __libc_start_main /build/buildd/eglibc-2.19/csu/libc-start.c:321:0

#20 0x10d8489 _start (/home/wilsonk/Downloads/Calypso/build/bin/ldc2+0x10d8489)

./BUILD: line 15: 31075 Aborted ldc2 -cM=zeromq.modulemap -cMH=../include/ -cpp-args -I../include -cpp-args -DZMQ_USE_EPOLL -g -L-L/home/wilsonk/Downloads/libzmq/lib -L-lsodium -L-lstdc++ main.d ../lib/*.o

* The reason I say ‘minimum’ zmq.hpp imported is because: if you ONLY have a main function and a modulemap imported, then there is a segfault in llvm::IntrusiveRefCntPtrclang::ASTContext::operator_() begin called from reclang->ASTUnit->getASTContext. Just making sure you are aware of this segfault when only importing a modulemap with no other imports. _

I will try to look at the ‘offset overflow’ error, I suppose, but I am not overly confident about being able to figure that one out. I have never worked with pch’s or module mapping, but no time like the present I guess :)

.

.

.

Now to one other thing….is there any way that you can contact me on IRC, please…..please, please, please, pretty please…or whatever I have to say??? This is so frustrating to only have contact through github!! We could have gone through this modulemap problem in under two minutes, if we were communicating directly, but it has been 24 hours instead and I got so frustrated that I went back to a different project last night, instead of banging my head against a wall over something that I knew was simple, but undocumented. I understand you are busy and don’t want to be pestered with a bunch of questions, but we really should have direct contact (when the time difference allows) to work more effectively together. If you just go to #ldc and personal message me with some obscure user name then nobody else will bother you, but at least we can resolve some issues quite a bit quicker that way. My username is wilsonk-laptop.

Thanks,

Kelly

From: Elie Morisse [mailto:[email protected]]
Sent: Tuesday, April 21, 2015 8:01 AM
To: Syniurge/Calypso
Cc: wilsonk
Subject: Re: [Calypso] Change StringLiteral mapping for wide chars and add a test case. (#11)

To load stdc.ctype it's import (C++) cpp.stdc.ctype; , and yes it's supposed to import global functions, variables, and anonymous tags (for example the anonymous enum in ctype.h).

—
Reply to this email directly or view it on GitHub #11 (comment) .

[Syniurge]

I had a shot at your test case and there were multiple issues, one is the already pushed one liner and the other one looks lke an odd DMD bug or limitation.

A test case in pure D that can't be compiled either by LDC or DMD:

import std.stdio, std.conv, std.string;

const (dchar)* testWideString(const (dchar)* defaultNotFound = "unknown"d)
{
    return defaultNotFound;
}

void main()
{
    writeln("testWideString() = ", testWideString());
}

The line that makes the implicit conversion fail is

if (!e->committed)

in cast.c(622). committed is true because the StringExp was semantic'd during ExpInitializer::semantic(). So doing an explicit cast like you did is the correct solution.

[Syniurge]

Works:

const (char)* testWideString(const (char)* defaultNotFound = "unknown")

Doesn't work:

const (char)* testWideString(const (char)* defaultNotFound = "unknown"c)

Error: cannot implicitly convert expression ("unknown"c) of type string to const(char)*

Rationale? 👀 I added the Issues tab so we can keep track of the DMD issues exposed by Calypso and maybe discuss them with DMD and LDC contributors to see whether they should be fixed or not.

[wilsonk]

Hello Elie,

That is an interesting limitation or bug…I assumed that conversion could be made. I suppose there may be some reason to limit that implicitly. At least the explicit cast works :)

I didn’t just use the one liner in my patch because I figured clang was doing some char sized specific optimizations (or something), when the strings weren’t wide. I assumed that is why they have a ->getString() method separate from getBytes(). A lot nicer looking code if we can just stick with the one liner, though.

Thanks,

Kelly

From: Elie Morisse [mailto:[email protected]]
Sent: Tuesday, April 21, 2015 12:13 PM
To: Syniurge/Calypso
Cc: wilsonk
Subject: Re: [Calypso] Change StringLiteral mapping for wide chars and add a test case. (#11)

I had a shot at your test case and there were multiple issues, one is the already pushd one liner and the other looks lke an odd DMD bug or limitation.

A simpler test case in pure D that can't be compiled either by LDC or DMD:

import std.stdio, std.conv, std.string;

const (dchar)* testWideString(const (dchar)* defaultNotFound = "unknown"d)
{
return defaultNotFound;
}

void main()
{
writeln("testWideString() = ", testWideString());
}

The line that makes the implicit conversion fail is

if (!e->committed)

in cast.c(622). committed is true because the StringExp was semantic'd during ExpInitializer::semantic(). So doing an explicit cast like you did is the correct solution.

—
Reply to this email directly or view it on GitHub #11 (comment) .

[Syniurge]

For wchar_t things seem to be sorted out, except that writeln doesn't display a string. Forcing wchar_t to map to either wchar or dchar doesn't help nor does to!dstring so I don't know.

About the module map support I didn't test it much, and I'm having trouble loading ctype.h alone, some odd issue with SourceManager's IDs.

It worked as part of a bigger PCH like the one for Ogre.h and yes the enum values could be used.

[Syniurge]

It works for me now, and functions are imported correctly:

writeln("_ISupper = ", _ISupper);
writeln("_IScntrl = ", _IScntrl);
writeln(cast(char)cpp.stdc.ctype.toupper('a'));
writeln(cast(char)tolower('B'));

_ISupper = 256
_IScntrl = 2
A
b