Some questions

I would like to learn to use ATS and also get some help using it
for some things.

  1. Can ATS be used to produce a stand alone C library?

This means: I will specify the functions which must be provided.
They will have extern “C” linkage.

I am thinking of Judy. Judy provides a cache tuned digital trie.
It is the fasted mutable associative store in existence which
provides both random and sequential access. There are 3 primary
data types: word -> bit, word -> word, and NTBS -> word.
[NTBS: null terminated string]

It also scales to terrabyte store without any issues.
All core operations on Judy are O(1) (O(N) for NTBS where
N is the string length).

Judy is slower at some sizes for random access compared
to hash tables (but hash tables can’t do sequential access).
OTOH Judy only handles word keys (hash is more general,
it handles anything with a comparison and hash function).

However the code is horrible, full of macros, and a lot of work would be
required to rewrite it in ATS. We’d use templates to replace the macros.
And we’d want to use the type system to prove correctness.

This would be an interesting job to do because Judy is the best.
And it isn’t used much for some reason (perhaps because the code
is unmaintainable :slight_smile:

Judy does have one downside in a modern context: it is not compatible
with a plugin garbage collector because it is a digital trie (keys are
split into non-contiguous byte streams which means some pointers
to objects will be invisible to a collector not trained in Judy).
My own GC knows about Judy though :slight_smile:

  1. I have another library which I would think is suitable for
    conversion to ATS: a precise garbage collector. Which uses
    Judy mentioned above. However it is currently a C++ library.
    The virtual dispatch is actually used as follows: the allocator
    used is abstract. The GC itself is split into an abstraction and
    implementation, the core implementation is not thread safe.
    A wrapper implementation is provides as well which is thread safe.

To convert to C is possible, just using a record of function pointers.

The GC is not entirely trivial. It’s a precise naive mark/sweep collector
driven by compiler generated tables. The main sweep mechanism
uses an array of offsets to find pointers. Judy arrays provide repetition
counts to allow variable length arrays to be handled.

When sweeping, the recursive descent posts pointers into a buffer
for later analysis until a fixed recursion depth is reached. Then the
scan falls out and the analysis routine runs through the buffer.
This mechanism is complex, a proof of correctness would be really
valuable.

This GC could also be used with ATS is the ATS compiler could be taught
how to generate the RTTI tables required to run the GC.

Unlike Judy, which it relies upon, the collector is not a very large
piece of code. The longest function is about one page.

  1. I have a number of other libraries which would be better written
    in ATS than C++ (and also some I do NOT have which I would like
    to write).

  2. More difficult: my compiler (Felix) generates C++ from a high level
    language. It is basically a souped up type safe macro processor
    which uses libraries to define Felix types and functions in terms of C++ types
    and functions. Then it churns away and emits C++.

It would be good to emit ATS as well. The ATS code would then be translated
by the ATS compiler into C which would then be compiled as C++ into a library
which is linked to the rest of the code. It’s essential that ATS compiler emits
C code which is also compliant C++ code (the primary requirement is to note
that C++ REQUIRES an explicit cast converting from a void * to another pointer
type, C doesn’t: there are some other issues: C++ has strongly typed enums
for example, C doesn’t).

This is a particularly hard project because it would be useless unless
Felix can be trained to emit suitable types and proof values.
[The Felix compiler doesn’t have to understand them, but it does have
to be able to sequence and compose them]

All these points above require one thing: ATS has to be able to generate

(a) libraries (not programs)
(b) functions with specified names and interfaces
© extern “C” wrappers
(d) C++ compilant C code

I believe ATS can do (a) and (b). © is a trivial mod.
(d) would require rewriting the compiler as required to ensure
casts are used where required, and enums are used in a suitable
way (enums in C++ are NOT integers).

john skaller
ska...@users.sourceforge.net
http://felix-lang.org

I would like to learn to use ATS and also get some help using it
for some things.

  1. Can ATS be used to produce a stand alone C library?

This means: I will specify the functions which must be provided.
They will have extern “C” linkage.

Yes. Here is an older post about it:
Redirecting to Google Groups

A lot of little projects with ‘MYPORTDIR’ subdirectories support this style.

For the C interface, see here:

I just checked here and it looks like this could stand for some updating:

I am thinking of Judy. Judy provides a cache tuned digital trie.
It is the fasted mutable associative store in existence which
provides both random and sequential access. There are 3 primary
data types: word → bit, word → word, and NTBS → word.
[NTBS: null terminated string]

It also scales to terrabyte store without any issues.
All core operations on Judy are O(1) (O(N) for NTBS where
N is the string length).

Judy is slower at some sizes for random access compared
to hash tables (but hash tables can’t do sequential access).
OTOH Judy only handles word keys (hash is more general,
it handles anything with a comparison and hash function).

However the code is horrible, full of macros, and a lot of work would be
required to rewrite it in ATS. We’d use templates to replace the macros.
And we’d want to use the type system to prove correctness.

This would be an interesting job to do because Judy is the best.
And it isn’t used much for some reason (perhaps because the code
is unmaintainable :slight_smile:

Judy does have one downside in a modern context: it is not compatible
with a plugin garbage collector because it is a digital trie (keys are
split into non-contiguous byte streams which means some pointers
to objects will be invisible to a collector not trained in Judy).
My own GC knows about Judy though :slight_smile:

  1. I have another library which I would think is suitable for
    conversion to ATS: a precise garbage collector. Which uses
    Judy mentioned above. However it is currently a C++ library.
    The virtual dispatch is actually used as follows: the allocator
    used is abstract. The GC itself is split into an abstraction and
    implementation, the core implementation is not thread safe.
    A wrapper implementation is provides as well which is thread safe.

To convert to C is possible, just using a record of function pointers.

The GC is not entirely trivial. It’s a precise naive mark/sweep collector
driven by compiler generated tables. The main sweep mechanism
uses an array of offsets to find pointers. Judy arrays provide repetition
counts to allow variable length arrays to be handled.

When sweeping, the recursive descent posts pointers into a buffer
for later analysis until a fixed recursion depth is reached. Then the
scan falls out and the analysis routine runs through the buffer.
This mechanism is complex, a proof of correctness would be really
valuable.

This GC could also be used with ATS is the ATS compiler could be taught
how to generate the RTTI tables required to run the GC.

Unlike Judy, which it relies upon, the collector is not a very large
piece of code. The longest function is about one page.

  1. I have a number of other libraries which would be better written
    in ATS than C++ (and also some I do NOT have which I would like
    to write).

  2. More difficult: my compiler (Felix) generates C++ from a high level
    language. It is basically a souped up type safe macro processor
    which uses libraries to define Felix types and functions in terms of C++
    types
    and functions. Then it churns away and emits C++.

It would be good to emit ATS as well. The ATS code would then be translated
by the ATS compiler into C which would then be compiled as C++ into a
library
which is linked to the rest of the code. It’s essential that ATS
compiler emits
C code which is also compliant C++ code (the primary requirement is to note
that C++ REQUIRES an explicit cast converting from a void * to another
pointer
type, C doesn’t: there are some other issues: C++ has strongly typed enums
for example, C doesn’t).

This is a particularly hard project because it would be useless unless
Felix can be trained to emit suitable types and proof values.
[The Felix compiler doesn’t have to understand them, but it does have
to be able to sequence and compose them]

All these points above require one thing: ATS has to be able to generate

(a) libraries (not programs)
(b) functions with specified names and interfaces
(c) extern “C” wrappers
(d) C++ compilant C code

I believe ATS can do (a) and (b). (c) is a trivial mod.
(d) would require rewriting the compiler as required to ensure
casts are used where required, and enums are used in a suitable
way (enums in C++ are NOT integers).


john skaller
ska...@users.sourceforge.net
http://felix-lang.org


You received this message because you are subscribed to the Google Groups
“ats-lang-users” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to ats-lang-user...@googlegroups.com.
To post to this group, send email to ats-lan...@googlegroups.com.
Visit this group at http://groups.google.com/group/ats-lang-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/ats-lang-users/825FAAD9-F0E2-4519-993C-B9D160EDA7CF%40users.sourceforge.net
.

Brandon Barker
brandon...@gmail.com

In the wiki article there is this, which I think is what you are referring to:

Make an ATS function available in C (can be implemented in ATS or C):

extern
fun myfun (n: int, res: int): int = “ext#myfun_in_c” (* call as myfun_in_c(int, int) in C *)

(* Have the C function name be the same as the ATS function name: )
extern
fun myfun (n: int, res: int): int = “ext#” (
call as myfun(int, int) in C *)

That’s not enough in C++. You have to say

// in header file:
extern “C” void name (int);

// in body code:
void name ( int x) { … }

But its WORSE. In both C and C++ you ALSO have to say something like:

#if !defined(FLX_STATIC_LINK) && FLX_WIN32
#define FLX_EXPORT __declspec(dllexport)
#define FLX_IMPORT __declspec(dllimport)
#else
#define FLX_EXPORT
#define FLX_IMPORT
#endif

#ifdef BUILD_RTL
#define RTL_EXTERN FLX_EXPORT
#else
#define RTL_EXTERN FLX_IMPORT
#endif

Here “BUILD_RTL” has to be supplied as a C/C++ compiler switch when building
the DLL and not otherwise. FLX_STATIC_LINK has to be supplied if compiling
for static linkage (and not otherwise).

Now the functions have to be declared like

RTL_EXTERN void flx_trace(flx_trace_t* tr,flx_range_srcref_t sr, char const *file, int line, char const *msg);

in C++ for a C++ function and

extern “C”

as well if you want to use C liker names. This is usually done like:

#ifdef __cplusplus /* support use by C++ code */
extern “C” {
#endif

RTL_EXTERN void flx_trace(flx_trace_t* tr,flx_range_srcref_t sr, char const *file, int line, char const *msg);

#ifdef __cplusplus
}
#endif

It used to be only Windows required dllimport/dllexport but this is NOT so any more.
Now GNU linkers (finally) have visibility control, you have to do it with all modern gcc
systems (i.e. all Unix, OSX and Linux) platforms too (unless you want to export everything!)

So clearly this is all non-trivial and it HAS to be supported by ATS compiler.
I’m pretty sure ATS compiler cannot do the DLL import/export at present.
In particular

(a) the ATS to C compiler has to generate the appropriate code AND

(b) the C/C++ compiler/linker toolchain invoked by ATS driver
program has to provide the required STATIC_LINK and BUILD_LIB switches
as well.

I don’t care about (b), I can organise C/C++ compile/link myself.

All this is in addition to ensuring the C code works under C++ compiler as well.
(no implicit casts from void * allowed, no assuming enums are ints, no funny business
with bool either … )

Even if you compile the library with C, the header files MUST work in C++
as well as C.

getting all this right is not hard but the method is NOT obvious: more than half the
open source projects in existence get it wrong.

The macro method for building DLLs above is the ONLY correct way to do it.

In turn this means when you’re translating a library from ATS to C you MUST
supply the build macro name to the ATS translator. Which I don’t think ATS
supports at present either.

So … I may be wrong … but I do not think ATS can generate libraries yet.

Just as an aside, although messy this is the correct protocol:

export sym; // in provider
import sym; // in consumer

C, as usual, gets everything wrong and tries to just use

extern sym;

for both purposes and that simply does not work. [In particular
the statement

extern int x;

has unspecified meaning: you can’t tell if it’s a declaration or definition,
so C’s hackery doesn’t even work in C]

john skaller
ska...@users.sourceforge.net
http://felix-lang.org

Some more posts on generating C libraries:

https://groups.google.com/forum/#!searchin/ats-lang-users/portable$20c$20ats/ats-lang-users/vw6VepY4x0k/YJXRIEx0w6UJ

https://groups.google.com/forum/#!searchin/ats-lang-users/portable$20c$20ats/ats-lang-users/NWSm3loV4RA/5dc4WbLxjskJ

I think the above links should address points (a), (b), and (c) in your
post.On Fri, Dec 12, 2014 at 10:01 PM, Brandon Barker brandon...@gmail.com wrote:

On Fri, Dec 12, 2014 at 9:15 PM, john skaller < ska...@users.sourceforge.net> wrote:

I would like to learn to use ATS and also get some help using it
for some things.

  1. Can ATS be used to produce a stand alone C library?

This means: I will specify the functions which must be provided.
They will have extern “C” linkage.

Yes. Here is an older post about it:

Redirecting to Google Groups

A lot of little projects with ‘MYPORTDIR’ subdirectories support this
style.

For the C interface, see here:

C interface · githwxi/ATS-Postiats Wiki · GitHub

I just checked here and it looks like this could stand for some updating:

Compiling ATS code · githwxi/ATS-Postiats Wiki · GitHub

I am thinking of Judy. Judy provides a cache tuned digital trie.
It is the fasted mutable associative store in existence which
provides both random and sequential access. There are 3 primary
data types: word → bit, word → word, and NTBS → word.
[NTBS: null terminated string]

It also scales to terrabyte store without any issues.
All core operations on Judy are O(1) (O(N) for NTBS where
N is the string length).

Judy is slower at some sizes for random access compared
to hash tables (but hash tables can’t do sequential access).
OTOH Judy only handles word keys (hash is more general,
it handles anything with a comparison and hash function).

However the code is horrible, full of macros, and a lot of work would be
required to rewrite it in ATS. We’d use templates to replace the macros.
And we’d want to use the type system to prove correctness.

This would be an interesting job to do because Judy is the best.
And it isn’t used much for some reason (perhaps because the code
is unmaintainable :slight_smile:

Judy does have one downside in a modern context: it is not compatible
with a plugin garbage collector because it is a digital trie (keys are
split into non-contiguous byte streams which means some pointers
to objects will be invisible to a collector not trained in Judy).
My own GC knows about Judy though :slight_smile:

  1. I have another library which I would think is suitable for
    conversion to ATS: a precise garbage collector. Which uses
    Judy mentioned above. However it is currently a C++ library.
    The virtual dispatch is actually used as follows: the allocator
    used is abstract. The GC itself is split into an abstraction and
    implementation, the core implementation is not thread safe.
    A wrapper implementation is provides as well which is thread safe.

To convert to C is possible, just using a record of function pointers.

The GC is not entirely trivial. It’s a precise naive mark/sweep collector
driven by compiler generated tables. The main sweep mechanism
uses an array of offsets to find pointers. Judy arrays provide repetition
counts to allow variable length arrays to be handled.

When sweeping, the recursive descent posts pointers into a buffer
for later analysis until a fixed recursion depth is reached. Then the
scan falls out and the analysis routine runs through the buffer.
This mechanism is complex, a proof of correctness would be really
valuable.

This GC could also be used with ATS is the ATS compiler could be taught
how to generate the RTTI tables required to run the GC.

Unlike Judy, which it relies upon, the collector is not a very large
piece of code. The longest function is about one page.

  1. I have a number of other libraries which would be better written
    in ATS than C++ (and also some I do NOT have which I would like
    to write).

  2. More difficult: my compiler (Felix) generates C++ from a high level
    language. It is basically a souped up type safe macro processor
    which uses libraries to define Felix types and functions in terms of C++
    types
    and functions. Then it churns away and emits C++.

It would be good to emit ATS as well. The ATS code would then be
translated
by the ATS compiler into C which would then be compiled as C++ into a
library
which is linked to the rest of the code. It’s essential that ATS
compiler emits
C code which is also compliant C++ code (the primary requirement is to
note
that C++ REQUIRES an explicit cast converting from a void * to another
pointer
type, C doesn’t: there are some other issues: C++ has strongly typed enums
for example, C doesn’t).

This is a particularly hard project because it would be useless unless
Felix can be trained to emit suitable types and proof values.
[The Felix compiler doesn’t have to understand them, but it does have
to be able to sequence and compose them]

All these points above require one thing: ATS has to be able to generate

(a) libraries (not programs)
(b) functions with specified names and interfaces
(c) extern “C” wrappers
(d) C++ compilant C code

I believe ATS can do (a) and (b). (c) is a trivial mod.
(d) would require rewriting the compiler as required to ensure
casts are used where required, and enums are used in a suitable
way (enums in C++ are NOT integers).


john skaller
ska...@users.sourceforge.net
http://felix-lang.org


You received this message because you are subscribed to the Google Groups
“ats-lang-users” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to ats-lang-user...@googlegroups.com.
To post to this group, send email to ats-lan...@googlegroups.com.
Visit this group at http://groups.google.com/group/ats-lang-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/ats-lang-users/825FAAD9-F0E2-4519-993C-B9D160EDA7CF%40users.sourceforge.net
.


Brandon Barker
brandon...@gmail.com

Brandon Barker
brandon...@gmail.com

If you search for ‘convoluted’, you will find the post.On Saturday, December 13, 2014 9:13:34 AM UTC-5, Brandon Barker wrote:

Sorry I misunderstood the issue, I have not done much C++ (though I would
be relatively happy to do). Barry, I also seem to have missed or not
remembered your post on the issue =/

On Saturday, December 13, 2014 12:06:44 AM UTC-5, Barry Schwartz wrote:

john skaller ska...@users.sourceforge.net skribis:

But its WORSE. In both C and C++ you ALSO have to say something like:

#if !defined(FLX_STATIC_LINK) && FLX_WIN32
#define FLX_EXPORT __declspec(dllexport)
#define FLX_IMPORT __declspec(dllimport)
#else
#define FLX_EXPORT
#define FLX_IMPORT
#endif

#ifdef BUILD_RTL
#define RTL_EXTERN FLX_EXPORT
#else
#define RTL_EXTERN FLX_IMPORT
#endif

etc.

I asked about this a few weeks ago. Yes, it can be done. It is not
done by using =“ext:” but by a more convoluted but otherwise better
method; I am not the one to discuss it, however.

Sorry I misunderstood the issue, I have not done much C++ (though I would
be relatively happy to do). Barry, I also seem to have missed or not
remembered your post on the issue =/On Saturday, December 13, 2014 12:06:44 AM UTC-5, Barry Schwartz wrote:

john skaller <ska...@users.sourceforge.net <javascript:>> skribis:

But its WORSE. In both C and C++ you ALSO have to say something like:

#if !defined(FLX_STATIC_LINK) && FLX_WIN32
#define FLX_EXPORT __declspec(dllexport)
#define FLX_IMPORT __declspec(dllimport)
#else
#define FLX_EXPORT
#define FLX_IMPORT
#endif

#ifdef BUILD_RTL
#define RTL_EXTERN FLX_EXPORT
#else
#define RTL_EXTERN FLX_IMPORT
#endif

etc.

I asked about this a few weeks ago. Yes, it can be done. It is not
done by using =“ext:” but by a more convoluted but otherwise better
method; I am not the one to discuss it, however.