In ATS2-contrib, there are also document/ and projects/, which are not
packed
into the current release of ATS2-contrib. There directories only contain
documentation,
which you can easily access on-line.On Wed, Oct 1, 2014 at 11:19 PM, ‘Yannick Duchêne’ via ats-lang-users < ats-lan...@googlegroups.com> wrote:
Le jeudi 2 octobre 2014 04:11:24 UTC+2, gmhwxi a écrit :
There is no default malloc/free pair. It has to be specified
by the programmer.
-DATS_MEMALLOC_LIBC: using libc malloc/free
-DATS_MEMALLOC_GCBDW: using Boehm GC
Here is an example showing how to supply a malloc/free pair of your own
Quick digression: seems I don’t find this in the contrib directory of my
installation. Should contrib from GitHub be installed instead of that of
from Sourceforge?
Yes. Sorry for not spelling it out to be clear. In the context of ATS I
thought that would have been the assumption.On Wednesday, October 1, 2014 11:10:09 AM UTC-7, gmhwxi wrote:
I assume that ADT means algebraic datatype in the given context.
Yes, you can have ADTs in pretty much any programming languages.
But the languages you mentioned above do not have a type system to
help the programmer use ADTs safely.
Is there a way to implement ADTs without relying on GC? Of course, such
a implementation should not leak memory. In ATS, you can use linear
datatypes
without GC and without fear of memory leaks.
On Wed, Oct 1, 2014 at 1:54 PM, Raoul Duke <rao...@gmail.com <javascript:> wrote:
However, the values to be matched against are usually compound values
that need to be created at run-time. I guess Haitao’s question is about
where
such compound values are stored and how they are freed.
i still don’t get it. i can have (things approximating at least) ADTs
in C/++/Obj-C, can i not?
(note: i’m not arguing for non-gc, i just do not yet understand the
issue here, it all seems confused and conflated and confabulated to
me.)
I have a working C implementation. It didn’t take much time but it is
bare-bones. The trick was to get out of the C world as soon as I can.
However if I can find a good more expressive replacement for C I feel a lot
more can be done to make the extension easier to use. I test drove some DLL
loading code using ATS at the time. While it is nice how easy it was to mix
with the C world the expressiveness level wasn’t much higher because I
refrained from using the more functional features. Do you have an example
of how you can implement something using all ATS features without regard to
memory allocation issues and then later on fixing the leaks while
maintaining the high level of expressiveness and readability? Showing some
before and after code can be very helpful for people to understand what the
expectation should be.On Wednesday, October 1, 2014 10:37:49 PM UTC-7, gmhwxi wrote:
I would not worry about freeing memory at the beginning. I would just
implement the extension and get
it work first. Only after getting a running implementation, I will start
to think about a way to get rid of memory leaks.
With the help of linear types, I feel pretty good about succeeding in
getting rid of leaks at the end. Juggling too many
things at once can easily make you mentally tired, losing the very
interest you need to go forward.
On Thu, Oct 2, 2014 at 12:20 AM, Haitao Zhang <zht...@gmail.com <javascript:>> wrote:
An extension could also call the host which might in turn call the
extension so it could be nested, but that may not be the norm. The problem
is like this:
We have a top level GC’ed interpreter that we want to write extensions
for. The large data structure is held in the extension memory with long
life cycles. The top level can call the extension to query about the data
held there. There are a lot of boiler plates to handle FFI calls to
translate data formats and parse call options etc. In the end some answer
is produced and send back to the top level caller. So the bulk of the
manually allocated linear objects are relatively static. I would like to
be able to use a higher level abstraction than what C could provide for
tasks that don’t involve malloc/free of the long-lived objects so that I
don’t have to worry about the little things. If the calls are not nested
then yes at the end of a call I can just batch free the non-linear objects
(those auto allocated for ADTs, closures etc) that will no longer be
needed. Only the linear objects need to persist between calls.
On Wed, Oct 1, 2014 at 8:59 PM, gmhwxi <gmh...@gmail.com <javascript:>> wrote:
My understanding is that you only need one allocator. Linear values can
be freed during a call to your library function and non-linear values
are all freed at once at the end of the call.
On Wednesday, October 1, 2014 11:51:03 PM UTC-4, H Zhang wrote:
Thanks for the clarification. I will give this approach some more
thoughts then. Since linear values can turn into non-linear values, the
non-linear allocator possibly needs to free the memory allocated by the
linear allocator, right? But the non-linear allocator would not know what
are pointers in the part of heap it manages, nor would it know if it is
safe to free what they point to?
Moving ahead one step, it would then be nice to distinguish linear and
non-linear allocations so we can have two heaps, one for the linear
structures and one for the nonlinear ones. Before type erasure ATS has
information on what allocations are linear, correct? Is there also a way to
guarantee that linear structures don’t contain pointers to the nonlinear
heap? This way we can also use linear types to return structures without
copying and/or store side effects, safer than using only CFFI? Not sure how
useful or feasible this is.
Linear values are often not freed; instead, they turn into
non-linear value
I think my answer was to the second part of your question (quoted
above). You cannot
have one heap for linear values and another one for non-linear values
because linear values
can be turned into non-linear ones.
Now I think I understand what you want to do. You can just write your
own heap allocator
(as was suggested in your first paragraph).
On Wednesday, October 1, 2014 10:14:27 PM UTC-4, H Zhang wrote:
Ah this is the thread (itself a continuation of a previous thread Redirecting to Google Groups
heap/ats-lang-users/jtg3f_P-xx0/ihW05rcULccJ
And your answer was that: “Linear values are often not freed;
instead, they turn into non-linear values”
So it seems the answer to my question would still be that it is still
not easy to use ADTs and closures.
On Wed, Oct 1, 2014 at 6:55 PM, Haitao Zhang zht...@gmail.com wrote:
That is good to know. Last time we talked (a few months back) I got
a different impression. So this means I can specify what malloc/free ATS
uses to compile ADT and closures? Or do I let them have the default and I
can specify a different set for the manually managed code? Last time I
looked at the instruction set macros of ATS I didn’t think that was
possible.
Yes, you can have different sets of malloc/free pairs as long as
you don’t use GC.
If you use GC, then you probably do not want to have different sets
of malloc/free pairs.
On Wednesday, October 1, 2014 7:27:41 PM UTC-4, H Zhang wrote:
Yes that is what regions analysis does. That is a good “poor
man”'s substitute for regions analysis but I think in ATS you can’t really
specify multiple mallocs (?). You would have to drop back to C to do that.
The problem with letting it dangle now and fix it later that
is: 1) I have
to remember doing that and also pick up what I was doing before
and 2)
assuming the tools give meaningful warnings so I can go right
to where the
issue is the “fix” would still pollute the code and make it
harder to read.
Really I should be able to say exe_name := file_name_stem +
“.exe” and
forget about it. I don’t care about the performance here nor
memory usage.
Just needs it to be safe. But it is a heavy price to pay if I
have to use
Python to get little niceties like this. If 90% of the cases
can be handled
automatically I don’t mind handling the cases that can’t be
resolved. Then
I can reserve the mental energy to use linear types on things
that really
matter to me performance wise (time and space).
In my working days as a C programmer of slow programs that could
eat
up memory freely, I used to use a little ‘reaping allocator’ I
wrote. Not at all complicated. Call it instead of malloc
directly, and
use it just like malloc, but the allocated pieces had a link
field in
them so you could free them all at once. So I’d start up the
allocator
at the beginning of routine and put a ‘reap’ at the end. In
between, I
could act somewhat as if I had a garbage collector.
Simple solutions often work fine.
(Python OTOH makes things so complicated that I wouldn’t trust a
proof
of correctness of some Python code if it existed, and don’t trust
tests.)
I once tried to juggle 3 oranges (or apples) in front of the students
taking my class
so as to show them how difficult it is. When programming, people tend to
juggle many
concepts mentally, complicating programming unnecessarily.
In ATS, you can always use a non-linear type first and then change it to a
linear one
later. This allows you to focus on what you are doing.On Wed, Oct 1, 2014 at 4:40 PM, H Zhang zht...@gmail.com wrote:
It is the mental distraction that is annoying. E.g. in C for something
trivial like changing the extension of a filename one needs to allocate a
string, pass it on and later remember to free it. That is just very
distracting from the logical flow. Not only that, it is very annoying to
read codes that are littered with things like that as it interferes with
following the programmer’s true intention.
On Wednesday, October 1, 2014 12:48:40 PM UTC-7, gmhwxi wrote:
without having to manually manage all the life cycles
Your assumption is that this must be very burdensome. But really? One
needs to actually
do it in order to get a real sense of it.
Think about a system that you want to implement. If there is something
that prevents you from
implementing the system, I bet it is unlikely due to the need to manually
manage something; it
usually happens at a much earlier stage.
On Wednesday, October 1, 2014 3:01:43 PM UTC-4, H Zhang wrote:
Rust is Apache/MIT licensed so that whatever they do should be okay to
borrow in ATS as well?
The way I think about the language mismatch problem, if we take data
structures as the central of focus of programming, is that managed
languages don’t let you control every detail of the data structure that is
most important to you, while c/c++ forces you to manage every little detail
that is unimportant to you. For the data structure that carries the whole
program state I want to be very careful about everything, but in any given
function I am only working on a small part of it I want to be able to use
abstract (not just primitive) temporary or not so temporary smallish
structures without having to manually manage all the life cycles. I don’t
know if Rust is It but I do think it is an important gap to address.
On Wednesday, October 1, 2014 10:35:18 AM UTC-7, gmhwxi wrote:
Interesting.
My guess is that some kind of (sophisticated) flow analysis is involved.
Is everything on stack?
Putting everything on stack would make it so difficult to do
general-purpose programming
even if it is still possible.
Why can’t ATS do something similar?
You mean to rely on flow analysis (instead of linear types) for memory
management?
Well, this has to do with my training. The lessons I learned from
automated theorem-proving
made me constantly try to avert complex algorithms of heavy heuristic
nature.
On Wednesday, October 1, 2014 12:20:34 PM UTC-4, H Zhang wrote:
Rust is able to do support ADT and pattern matching without requiring
the use of GC. Does anyone know how that is achieved? Is everything on
stack? Why can’t ATS do something similar?
without having to manually manage all the life cycles
Your assumption is that this must be very burdensome. But really? One needs
to actually
do it in order to get a real sense of it.
Think about a system that you want to implement. If there is something that
prevents you from
implementing the system, I bet it is unlikely due to the need to manually
manage something; it
usually happens at a much earlier stage.On Wednesday, October 1, 2014 3:01:43 PM UTC-4, H Zhang wrote:
Rust is Apache/MIT licensed so that whatever they do should be okay to
borrow in ATS as well?
The way I think about the language mismatch problem, if we take data
structures as the central of focus of programming, is that managed
languages don’t let you control every detail of the data structure that is
most important to you, while c/c++ forces you to manage every little detail
that is unimportant to you. For the data structure that carries the whole
program state I want to be very careful about everything, but in any given
function I am only working on a small part of it I want to be able to use
abstract (not just primitive) temporary or not so temporary smallish
structures without having to manually manage all the life cycles. I don’t
know if Rust is It but I do think it is an important gap to address.
On Wednesday, October 1, 2014 10:35:18 AM UTC-7, gmhwxi wrote:
Interesting.
My guess is that some kind of (sophisticated) flow analysis is involved.
Is everything on stack?
Putting everything on stack would make it so difficult to do
general-purpose programming
even if it is still possible.
Why can’t ATS do something similar?
You mean to rely on flow analysis (instead of linear types) for memory
management?
Well, this has to do with my training. The lessons I learned from
automated theorem-proving
made me constantly try to avert complex algorithms of heavy heuristic
nature.
On Wednesday, October 1, 2014 12:20:34 PM UTC-4, H Zhang wrote:
Rust is able to do support ADT and pattern matching without requiring
the use of GC. Does anyone know how that is achieved? Is everything on
stack? Why can’t ATS do something similar?
My understanding is that you only need one allocator. Linear values can be
freed during a call to your library function and non-linear values
are all freed at once at the end of the call.On Wednesday, October 1, 2014 11:51:03 PM UTC-4, H Zhang wrote:
Thanks for the clarification. I will give this approach some more thoughts
then. Since linear values can turn into non-linear values, the non-linear
allocator possibly needs to free the memory allocated by the linear
allocator, right? But the non-linear allocator would not know what are
pointers in the part of heap it manages, nor would it know if it is safe to
free what they point to?
On Wed, Oct 1, 2014 at 7:33 PM, gmhwxi <gmh...@gmail.com <javascript:>> wrote:
Moving ahead one step, it would then be nice to distinguish linear and
non-linear allocations so we can have two heaps, one for the linear
structures and one for the nonlinear ones. Before type erasure ATS has
information on what allocations are linear, correct? Is there also a way to
guarantee that linear structures don’t contain pointers to the nonlinear
heap? This way we can also use linear types to return structures without
copying and/or store side effects, safer than using only CFFI? Not sure how
useful or feasible this is.
Linear values are often not freed; instead, they turn into non-linear
value
I think my answer was to the second part of your question (quoted above).
You cannot
have one heap for linear values and another one for non-linear values
because linear values
can be turned into non-linear ones.
Now I think I understand what you want to do. You can just write your own
heap allocator
(as was suggested in your first paragraph).
On Wednesday, October 1, 2014 10:14:27 PM UTC-4, H Zhang wrote:
Ah this is the thread (itself a continuation of a previous thread Redirecting to Google Groups
users/heap/ats-lang-users/jtg3f_P-xx0/ihW05rcULccJ
And your answer was that: “Linear values are often not freed; instead,
they turn into non-linear values”
So it seems the answer to my question would still be that it is still
not easy to use ADTs and closures.
On Wed, Oct 1, 2014 at 6:55 PM, Haitao Zhang zht...@gmail.com wrote:
That is good to know. Last time we talked (a few months back) I got a
different impression. So this means I can specify what malloc/free ATS uses
to compile ADT and closures? Or do I let them have the default and I can
specify a different set for the manually managed code? Last time I looked
at the instruction set macros of ATS I didn’t think that was possible.
Yes, you can have different sets of malloc/free pairs as long as you
don’t use GC.
If you use GC, then you probably do not want to have different sets of
malloc/free pairs.
On Wednesday, October 1, 2014 7:27:41 PM UTC-4, H Zhang wrote:
Yes that is what regions analysis does. That is a good “poor man”'s
substitute for regions analysis but I think in ATS you can’t really specify
multiple mallocs (?). You would have to drop back to C to do that.
The problem with letting it dangle now and fix it later that is:
I have
to remember doing that and also pick up what I was doing before
and 2)
assuming the tools give meaningful warnings so I can go right to
where the
issue is the “fix” would still pollute the code and make it harder
to read.
Really I should be able to say exe_name := file_name_stem + “.exe”
and
forget about it. I don’t care about the performance here nor
memory usage.
Just needs it to be safe. But it is a heavy price to pay if I have
to use
Python to get little niceties like this. If 90% of the cases can
be handled
automatically I don’t mind handling the cases that can’t be
resolved. Then
I can reserve the mental energy to use linear types on things that
really
matter to me performance wise (time and space).
In my working days as a C programmer of slow programs that could eat
up memory freely, I used to use a little ‘reaping allocator’ I
wrote. Not at all complicated. Call it instead of malloc directly,
and
use it just like malloc, but the allocated pieces had a link field in
them so you could free them all at once. So I’d start up the
allocator
at the beginning of routine and put a ‘reap’ at the end. In between,
I
could act somewhat as if I had a garbage collector.
Simple solutions often work fine.
(Python OTOH makes things so complicated that I wouldn’t trust a
proof
of correctness of some Python code if it existed, and don’t trust
tests.)
I assume that ADT means algebraic datatype in the given context.
Yes, you can have ADTs in pretty much any programming languages.
But the languages you mentioned above do not have a type system to
help the programmer use ADTs safely.
Is there a way to implement ADTs without relying on GC? Of course, such
a implementation should not leak memory. In ATS, you can use linear
datatypes
without GC and without fear of memory leaks.On Wed, Oct 1, 2014 at 1:54 PM, Raoul Duke rao...@gmail.com wrote:
However, the values to be matched against are usually compound values
that need to be created at run-time. I guess Haitao’s question is about
where
such compound values are stored and how they are freed.
i still don’t get it. i can have (things approximating at least) ADTs
in C/++/Obj-C, can i not?
(note: i’m not arguing for non-gc, i just do not yet understand the
issue here, it all seems confused and conflated and confabulated to
me.)
It is the mental distraction that is annoying. E.g. in C for something
trivial like changing the extension of a filename one needs to allocate a
string, pass it on and later remember to free it. That is just very
distracting from the logical flow. Not only that, it is very annoying to
read codes that are littered with things like that as it interferes with
following the programmer’s true intention.
if you could snap your fingers and magically make something that
works, what do you think it would look like.
e.g. even with GC one still has to have something like IDisposeable.
personally i day-dream of a hybrid RC+GC that would be the best of all
possible worlds. there’s quite a bit of interesting research on that,
but nothing productized enough for e.g. me to easily use it for making
video games across platforms. like, i guess i’d want something that
matches Boehm API since it seems like a lot of projects know about
that at least, but does a better job than Boehm. (not that i
personally have metrics on Boehm, i’m just going based on what some
people i think i trust have said about it.
An extension could also call the host which might in turn call the
extension so it could be nested, but that may not be the norm. The problem
is like this:
We have a top level GC’ed interpreter that we want to write extensions for.
The large data structure is held in the extension memory with long life
cycles. The top level can call the extension to query about the data held
there. There are a lot of boiler plates to handle FFI calls to translate
data formats and parse call options etc. In the end some answer is produced
and send back to the top level caller. So the bulk of the manually
allocated linear objects are relatively static. I would like to be able to
use a higher level abstraction than what C could provide for tasks that
don’t involve malloc/free of the long-lived objects so that I don’t have to
worry about the little things. If the calls are not nested then yes at the
end of a call I can just batch free the non-linear objects (those auto
allocated for ADTs, closures etc) that will no longer be needed. Only the
linear objects need to persist between calls.On Wed, Oct 1, 2014 at 8:59 PM, gmhwxi gmh...@gmail.com wrote:
My understanding is that you only need one allocator. Linear values can be
freed during a call to your library function and non-linear values
are all freed at once at the end of the call.
On Wednesday, October 1, 2014 11:51:03 PM UTC-4, H Zhang wrote:
Thanks for the clarification. I will give this approach some more
thoughts then. Since linear values can turn into non-linear values, the
non-linear allocator possibly needs to free the memory allocated by the
linear allocator, right? But the non-linear allocator would not know what
are pointers in the part of heap it manages, nor would it know if it is
safe to free what they point to?
Moving ahead one step, it would then be nice to distinguish linear and
non-linear allocations so we can have two heaps, one for the linear
structures and one for the nonlinear ones. Before type erasure ATS has
information on what allocations are linear, correct? Is there also a way to
guarantee that linear structures don’t contain pointers to the nonlinear
heap? This way we can also use linear types to return structures without
copying and/or store side effects, safer than using only CFFI? Not sure how
useful or feasible this is.
Linear values are often not freed; instead, they turn into non-linear
value
I think my answer was to the second part of your question (quoted
above). You cannot
have one heap for linear values and another one for non-linear values
because linear values
can be turned into non-linear ones.
Now I think I understand what you want to do. You can just write your
own heap allocator
(as was suggested in your first paragraph).
On Wednesday, October 1, 2014 10:14:27 PM UTC-4, H Zhang wrote:
Ah this is the thread (itself a continuation of a previous thread Redirecting to Google Groups
heap/ats-lang-users/jtg3f_P-xx0/ihW05rcULccJ
And your answer was that: “Linear values are often not freed; instead,
they turn into non-linear values”
So it seems the answer to my question would still be that it is still
not easy to use ADTs and closures.
On Wed, Oct 1, 2014 at 6:55 PM, Haitao Zhang zht...@gmail.com wrote:
That is good to know. Last time we talked (a few months back) I got a
different impression. So this means I can specify what malloc/free ATS uses
to compile ADT and closures? Or do I let them have the default and I can
specify a different set for the manually managed code? Last time I looked
at the instruction set macros of ATS I didn’t think that was possible.
Yes, you can have different sets of malloc/free pairs as long as you
don’t use GC.
If you use GC, then you probably do not want to have different sets
of malloc/free pairs.
On Wednesday, October 1, 2014 7:27:41 PM UTC-4, H Zhang wrote:
Yes that is what regions analysis does. That is a good “poor man”'s
substitute for regions analysis but I think in ATS you can’t really specify
multiple mallocs (?). You would have to drop back to C to do that.
The problem with letting it dangle now and fix it later that is:
I have
to remember doing that and also pick up what I was doing before
and 2)
assuming the tools give meaningful warnings so I can go right to
where the
issue is the “fix” would still pollute the code and make it
harder to read.
Really I should be able to say exe_name := file_name_stem +
“.exe” and
forget about it. I don’t care about the performance here nor
memory usage.
Just needs it to be safe. But it is a heavy price to pay if I
have to use
Python to get little niceties like this. If 90% of the cases can
be handled
automatically I don’t mind handling the cases that can’t be
resolved. Then
I can reserve the mental energy to use linear types on things
that really
matter to me performance wise (time and space).
In my working days as a C programmer of slow programs that could eat
up memory freely, I used to use a little ‘reaping allocator’ I
wrote. Not at all complicated. Call it instead of malloc directly,
and
use it just like malloc, but the allocated pieces had a link field
in
them so you could free them all at once. So I’d start up the
allocator
at the beginning of routine and put a ‘reap’ at the end. In
between, I
could act somewhat as if I had a garbage collector.
Simple solutions often work fine.
(Python OTOH makes things so complicated that I wouldn’t trust a
proof
of correctness of some Python code if it existed, and don’t trust
tests.)
My guess is that some kind of (sophisticated) flow analysis is involved.
Is everything on stack?
Putting everything on stack would make it so difficult to do
general-purpose programming
even if it is still possible.
Why can’t ATS do something similar?
You mean to rely on flow analysis (instead of linear types) for memory
management?
Well, this has to do with my training. The lessons I learned from automated
theorem-proving
made me constantly try to avert complex algorithms of heavy heuristic
nature.On Wednesday, October 1, 2014 12:20:34 PM UTC-4, H Zhang wrote:
Rust is able to do support ADT and pattern matching without requiring the
use of GC. Does anyone know how that is achieved? Is everything on stack?
Why can’t ATS do something similar?
Yes. You can also use the shorter keyword ‘datavtype’ for declaring a
linear datatype.On Wed, Oct 1, 2014 at 3:05 PM, Shea Levy sh...@shealevy.com wrote:
Are “linear datatypes” dataviewtypes?
~Shea
On Wed, Oct 01, 2014 at 02:10:09PM -0400, Hongwei Xi wrote:
I assume that ADT means algebraic datatype in the given context.
Yes, you can have ADTs in pretty much any programming languages.
But the languages you mentioned above do not have a type system to
help the programmer use ADTs safely.
Is there a way to implement ADTs without relying on GC? Of course, such
a implementation should not leak memory. In ATS, you can use linear
datatypes
without GC and without fear of memory leaks.
On Wed, Oct 1, 2014 at 1:54 PM, Raoul Duke rao...@gmail.com wrote:
However, the values to be matched against are usually compound values
that need to be created at run-time. I guess Haitao’s question is
about
where
such compound values are stored and how they are freed.
i still don’t get it. i can have (things approximating at least) ADTs
in C/++/Obj-C, can i not?
(note: i’m not arguing for non-gc, i just do not yet understand the
issue here, it all seems confused and conflated and confabulated to
me.)
Indeed. This kind of gradual software development is what I think a great
strength of ATS.On Wed, Oct 1, 2014 at 5:48 PM, Raoul Duke rao...@gmail.com wrote:
In ATS, you can always use a non-linear type first and then change it to
a
linear one
later. This allows you to focus on what you are doing.
do i get it right that you can start with Boehm first, too? as the
most easy route to Getting Something Working. and then easily
(easily?) migrate to statically tracked manual memory management?
I have a working C implementation. It didn’t take much time but it is
bare-bones. The trick was to get out of the C world as soon as I can.
However if I can find a good more expressive replacement for C I feel a lot
more can be done to make the extension easier to use. I test drove some DLL
loading code using ATS at the time. While it is nice how easy it was to mix
with the C world the expressiveness level wasn’t much higher because I
refrained from using the more functional features. Do you have an example
of how you can implement something using all ATS features without regard to
memory allocation issues and then later on fixing the leaks while
maintaining the high level of expressiveness and readability? Showing some
before and after code can be very helpful for people to understand what the
expectation should be.
On Wednesday, October 1, 2014 10:37:49 PM UTC-7, gmhwxi wrote:
I would not worry about freeing memory at the beginning. I would just
implement the extension and get
it work first. Only after getting a running implementation, I will start
to think about a way to get rid of memory leaks.
With the help of linear types, I feel pretty good about succeeding in
getting rid of leaks at the end. Juggling too many
things at once can easily make you mentally tired, losing the very
interest you need to go forward.
On Thu, Oct 2, 2014 at 12:20 AM, Haitao Zhang zht...@gmail.com wrote:
An extension could also call the host which might in turn call the
extension so it could be nested, but that may not be the norm. The problem
is like this:
We have a top level GC’ed interpreter that we want to write extensions
for. The large data structure is held in the extension memory with long
life cycles. The top level can call the extension to query about the data
held there. There are a lot of boiler plates to handle FFI calls to
translate data formats and parse call options etc. In the end some answer
is produced and send back to the top level caller. So the bulk of the
manually allocated linear objects are relatively static. I would like to
be able to use a higher level abstraction than what C could provide for
tasks that don’t involve malloc/free of the long-lived objects so that I
don’t have to worry about the little things. If the calls are not nested
then yes at the end of a call I can just batch free the non-linear objects
(those auto allocated for ADTs, closures etc) that will no longer be
needed. Only the linear objects need to persist between calls.
My understanding is that you only need one allocator. Linear values can
be freed during a call to your library function and non-linear values
are all freed at once at the end of the call.
On Wednesday, October 1, 2014 11:51:03 PM UTC-4, H Zhang wrote:
Thanks for the clarification. I will give this approach some more
thoughts then. Since linear values can turn into non-linear values, the
non-linear allocator possibly needs to free the memory allocated by the
linear allocator, right? But the non-linear allocator would not know what
are pointers in the part of heap it manages, nor would it know if it is
safe to free what they point to?
Moving ahead one step, it would then be nice to distinguish linear
and non-linear allocations so we can have two heaps, one for the linear
structures and one for the nonlinear ones. Before type erasure ATS has
information on what allocations are linear, correct? Is there also a way to
guarantee that linear structures don’t contain pointers to the nonlinear
heap? This way we can also use linear types to return structures without
copying and/or store side effects, safer than using only CFFI? Not sure how
useful or feasible this is.
Linear values are often not freed; instead, they turn into
non-linear value
I think my answer was to the second part of your question (quoted
above). You cannot
have one heap for linear values and another one for non-linear values
because linear values
can be turned into non-linear ones.
Now I think I understand what you want to do. You can just write your
own heap allocator
(as was suggested in your first paragraph).
On Wednesday, October 1, 2014 10:14:27 PM UTC-4, H Zhang wrote:
Ah this is the thread (itself a continuation of a previous thread Redirecting to Google Groups
ap/ats-lang-users/jtg3f_P-xx0/ihW05rcULccJ
And your answer was that: “Linear values are often not freed;
instead, they turn into non-linear values”
So it seems the answer to my question would still be that it is
still not easy to use ADTs and closures.
On Wed, Oct 1, 2014 at 6:55 PM, Haitao Zhang zht...@gmail.com wrote:
That is good to know. Last time we talked (a few months back) I got
a different impression. So this means I can specify what malloc/free ATS
uses to compile ADT and closures? Or do I let them have the default and I
can specify a different set for the manually managed code? Last time I
looked at the instruction set macros of ATS I didn’t think that was
possible.
Yes, you can have different sets of malloc/free pairs as long as
you don’t use GC.
If you use GC, then you probably do not want to have different
sets of malloc/free pairs.
On Wednesday, October 1, 2014 7:27:41 PM UTC-4, H Zhang wrote:
Yes that is what regions analysis does. That is a good “poor
man”'s substitute for regions analysis but I think in ATS you can’t really
specify multiple mallocs (?). You would have to drop back to C to do that.
The problem with letting it dangle now and fix it later that
is: 1) I have
to remember doing that and also pick up what I was doing
before and 2)
assuming the tools give meaningful warnings so I can go right
to where the
issue is the “fix” would still pollute the code and make it
harder to read.
Really I should be able to say exe_name := file_name_stem +
“.exe” and
forget about it. I don’t care about the performance here nor
memory usage.
Just needs it to be safe. But it is a heavy price to pay if I
have to use
Python to get little niceties like this. If 90% of the cases
can be handled
automatically I don’t mind handling the cases that can’t be
resolved. Then
I can reserve the mental energy to use linear types on things
that really
matter to me performance wise (time and space).
In my working days as a C programmer of slow programs that could
eat
up memory freely, I used to use a little ‘reaping allocator’ I
wrote. Not at all complicated. Call it instead of malloc
directly, and
use it just like malloc, but the allocated pieces had a link
field in
them so you could free them all at once. So I’d start up the
allocator
at the beginning of routine and put a ‘reap’ at the end. In
between, I
could act somewhat as if I had a garbage collector.
Simple solutions often work fine.
(Python OTOH makes things so complicated that I wouldn’t trust a
proof
of correctness of some Python code if it existed, and don’t trust
tests.)
The problem is that you don’t get to see the process of doing the
transition; you
can only see the end product of the transition.
Actually, a lot of code in libats went through the non-linear to linear
transition.
For instance, the linear version of AVL tree implementation is entirely
based on the
non-linear version. I emphasized this point explicitly in my ATS-intro book:
I have a working C implementation. It didn’t take much time but it is
bare-bones. The trick was to get out of the C world as soon as I can.
However if I can find a good more expressive replacement for C I feel a lot
more can be done to make the extension easier to use. I test drove some DLL
loading code using ATS at the time. While it is nice how easy it was to mix
with the C world the expressiveness level wasn’t much higher because I
refrained from using the more functional features. Do you have an example
of how you can implement something using all ATS features without regard to
memory allocation issues and then later on fixing the leaks while
maintaining the high level of expressiveness and readability? Showing some
before and after code can be very helpful for people to understand what the
expectation should be.
On Wednesday, October 1, 2014 10:37:49 PM UTC-7, gmhwxi wrote:
I would not worry about freeing memory at the beginning. I would just
implement the extension and get
it work first. Only after getting a running implementation, I will start
to think about a way to get rid of memory leaks.
With the help of linear types, I feel pretty good about succeeding in
getting rid of leaks at the end. Juggling too many
things at once can easily make you mentally tired, losing the very
interest you need to go forward.
On Thu, Oct 2, 2014 at 12:20 AM, Haitao Zhang zht...@gmail.com wrote:
An extension could also call the host which might in turn call the
extension so it could be nested, but that may not be the norm. The problem
is like this:
We have a top level GC’ed interpreter that we want to write extensions
for. The large data structure is held in the extension memory with long
life cycles. The top level can call the extension to query about the data
held there. There are a lot of boiler plates to handle FFI calls to
translate data formats and parse call options etc. In the end some answer
is produced and send back to the top level caller. So the bulk of the
manually allocated linear objects are relatively static. I would like to
be able to use a higher level abstraction than what C could provide for
tasks that don’t involve malloc/free of the long-lived objects so that I
don’t have to worry about the little things. If the calls are not nested
then yes at the end of a call I can just batch free the non-linear objects
(those auto allocated for ADTs, closures etc) that will no longer be
needed. Only the linear objects need to persist between calls.
My understanding is that you only need one allocator. Linear values can
be freed during a call to your library function and non-linear values
are all freed at once at the end of the call.
On Wednesday, October 1, 2014 11:51:03 PM UTC-4, H Zhang wrote:
Thanks for the clarification. I will give this approach some more
thoughts then. Since linear values can turn into non-linear values, the
non-linear allocator possibly needs to free the memory allocated by the
linear allocator, right? But the non-linear allocator would not know what
are pointers in the part of heap it manages, nor would it know if it is
safe to free what they point to?
Moving ahead one step, it would then be nice to distinguish linear
and non-linear allocations so we can have two heaps, one for the linear
structures and one for the nonlinear ones. Before type erasure ATS has
information on what allocations are linear, correct? Is there also a way to
guarantee that linear structures don’t contain pointers to the nonlinear
heap? This way we can also use linear types to return structures without
copying and/or store side effects, safer than using only CFFI? Not sure how
useful or feasible this is.
Linear values are often not freed; instead, they turn into
non-linear value
I think my answer was to the second part of your question (quoted
above). You cannot
have one heap for linear values and another one for non-linear values
because linear values
can be turned into non-linear ones.
Now I think I understand what you want to do. You can just write your
own heap allocator
(as was suggested in your first paragraph).
On Wednesday, October 1, 2014 10:14:27 PM UTC-4, H Zhang wrote:
Ah this is the thread (itself a continuation of a previous thread Redirecting to Google Groups
ap/ats-lang-users/jtg3f_P-xx0/ihW05rcULccJ
And your answer was that: “Linear values are often not freed;
instead, they turn into non-linear values”
So it seems the answer to my question would still be that it is
still not easy to use ADTs and closures.
On Wed, Oct 1, 2014 at 6:55 PM, Haitao Zhang zht...@gmail.com wrote:
That is good to know. Last time we talked (a few months back) I got
a different impression. So this means I can specify what malloc/free ATS
uses to compile ADT and closures? Or do I let them have the default and I
can specify a different set for the manually managed code? Last time I
looked at the instruction set macros of ATS I didn’t think that was
possible.
Yes, you can have different sets of malloc/free pairs as long as
you don’t use GC.
If you use GC, then you probably do not want to have different
sets of malloc/free pairs.
On Wednesday, October 1, 2014 7:27:41 PM UTC-4, H Zhang wrote:
Yes that is what regions analysis does. That is a good “poor
man”'s substitute for regions analysis but I think in ATS you can’t really
specify multiple mallocs (?). You would have to drop back to C to do that.
The problem with letting it dangle now and fix it later that
is: 1) I have
to remember doing that and also pick up what I was doing
before and 2)
assuming the tools give meaningful warnings so I can go right
to where the
issue is the “fix” would still pollute the code and make it
harder to read.
Really I should be able to say exe_name := file_name_stem +
“.exe” and
forget about it. I don’t care about the performance here nor
memory usage.
Just needs it to be safe. But it is a heavy price to pay if I
have to use
Python to get little niceties like this. If 90% of the cases
can be handled
automatically I don’t mind handling the cases that can’t be
resolved. Then
I can reserve the mental energy to use linear types on things
that really
matter to me performance wise (time and space).
In my working days as a C programmer of slow programs that could
eat
up memory freely, I used to use a little ‘reaping allocator’ I
wrote. Not at all complicated. Call it instead of malloc
directly, and
use it just like malloc, but the allocated pieces had a link
field in
them so you could free them all at once. So I’d start up the
allocator
at the beginning of routine and put a ‘reap’ at the end. In
between, I
could act somewhat as if I had a garbage collector.
Simple solutions often work fine.
(Python OTOH makes things so complicated that I wouldn’t trust a
proof
of correctness of some Python code if it existed, and don’t trust
tests.)
part. ML Kit tried but regions management didn’t work 100%. I am still
trying to figure out how Rust is doing it.
i think Rust tried and claimed optimistic success with more than one
thing only to back off and go another way later. if i actually
understood the issue i assume i/one could google up answers with the
right keywords, no?
That requires GC at some stage or “fixing” it manually. See Hongwei’s
earlier response:
"
Is there a way to implement ADTs without relying on GC? Of course, such
a implementation should not leak memory. In ATS, you can use linear
datatypes
without GC and without fear of memory leaks.
"On Wed, Oct 1, 2014 at 3:50 PM, Shea Levy sh...@shealevy.com wrote:
If you want to just do exe_name := file_name_step + “.exe” and not have
to worry about it, then what’s wrong exactly with using normal ADTs?
~Shea
On Wed, Oct 01, 2014 at 03:41:40PM -0700, H Zhang wrote:
The problem with letting it dangle now and fix it later that is: 1) I
have
to remember doing that and also pick up what I was doing before and 2)
assuming the tools give meaningful warnings so I can go right to where
the
issue is the “fix” would still pollute the code and make it harder to
read.
Really I should be able to say exe_name := file_name_stem + “.exe” and
forget about it. I don’t care about the performance here nor memory
usage.
Just needs it to be safe. But it is a heavy price to pay if I have to use
Python to get little niceties like this. If 90% of the cases can be
handled
automatically I don’t mind handling the cases that can’t be resolved.
Then
I can reserve the mental energy to use linear types on things that really
matter to me performance wise (time and space).
On Wednesday, October 1, 2014 2:45:06 PM UTC-7, gmhwxi wrote:
It is the mental distraction that is annoying.
I once tried to juggle 3 oranges (or apples) in front of the students
taking my class
so as to show them how difficult it is. When programming, people tend
to
juggle many
concepts mentally, complicating programming unnecessarily.
In ATS, you can always use a non-linear type first and then change it
to a
linear one
later. This allows you to focus on what you are doing.
On Wed, Oct 1, 2014 at 4:40 PM, H Zhang <zht...@gmail.com <javascript:>> wrote:
It is the mental distraction that is annoying. E.g. in C for something
trivial like changing the extension of a filename one needs to
allocate a
string, pass it on and later remember to free it. That is just very
distracting from the logical flow. Not only that, it is very annoying
to
read codes that are littered with things like that as it interferes
with
following the programmer’s true intention.
On Wednesday, October 1, 2014 12:48:40 PM UTC-7, gmhwxi wrote:
without having to manually manage all the life cycles
Your assumption is that this must be very burdensome. But really? One
needs to actually
do it in order to get a real sense of it.
Think about a system that you want to implement. If there is
something
that prevents you from
implementing the system, I bet it is unlikely due to the need to
manually manage something; it
usually happens at a much earlier stage.
On Wednesday, October 1, 2014 3:01:43 PM UTC-4, H Zhang wrote:
Rust is Apache/MIT licensed so that whatever they do should be okay
to
borrow in ATS as well?
The way I think about the language mismatch problem, if we take data
structures as the central of focus of programming, is that managed
languages don’t let you control every detail of the data structure
that is
most important to you, while c/c++ forces you to manage every
little detail
that is unimportant to you. For the data structure that carries the
whole
program state I want to be very careful about everything, but in
any given
function I am only working on a small part of it I want to be able
to use
abstract (not just primitive) temporary or not so temporary smallish
structures without having to manually manage all the life cycles. I
don’t
know if Rust is It but I do think it is an important gap to address.
On Wednesday, October 1, 2014 10:35:18 AM UTC-7, gmhwxi wrote:
Interesting.
My guess is that some kind of (sophisticated) flow analysis is
involved.
Is everything on stack?
Putting everything on stack would make it so difficult to do
general-purpose programming
even if it is still possible.
Why can’t ATS do something similar?
You mean to rely on flow analysis (instead of linear types) for
memory
management?
Well, this has to do with my training. The lessons I learned from
automated theorem-proving
made me constantly try to avert complex algorithms of heavy
heuristic
nature.
On Wednesday, October 1, 2014 12:20:34 PM UTC-4, H Zhang wrote:
Rust is able to do support ADT and pattern matching without
requiring
the use of GC. Does anyone know how that is achieved? Is
everything on
stack? Why can’t ATS do something similar?
Haitao
–
You received this message because you are subscribed to the Google
Groups
“ats-lang-users” group.
To unsubscribe from this group and stop receiving emails from it,
send an
email to ats-lang...@googlegroups.com <javascript:>.
To post to this group, send email to ats-l...@googlegroups.com
<javascript:>.
Visit this group at http://groups.google.com/group/ats-lang-users.
To view this discussion on the web visit
I’m disabled now, but when I was working I’d often write code for some
string-parsing task in Icon and then translate the result to C. Hardly
any difference in what the code did, but in Icon it was easier to
think.
now if only there were a compiler that could do that for ya…
The typechecker of ATS will enforce type-correctness,
which will get rid of a lot of stupid bugs.
I have put it this way:
Most programming errors are type errors in some language that people
aren’t using.
nice!
on the whole i mostly heartily concur with that way of thinking.
(although the time i saw a usenet post where somebody was trying to
explain how to use monads in haskell to reverse a string (in place, i
guess it must have been) was just horrible, vile, evil stuff.
