Non-standard memory allocator

Google Group
1

Hello,

Is there a way to customize memory allocator in ATS?

For example, in Apache and Postgres servers there are a memory pools to be
used in the context of the request. They are simpler (and more efficient)
than standard malloc for this particular task. GC is trivial is this case.

Thanks,

Valentyn.

2

How do C programs do in this case?

3

Hi Chris,

the question was how to do that - so that non-linear types (closures,
pairs, lists, arrays…) would use palloc instead of maloc with GC.

Thanks,

Valentyn.

4

Hello Valentyn,

It is certainly possible. If you use GC in the same program, then you will
also have to cooperate with it. That is, register the allocated memory with
ATS_GC_MARKROOT as GC roots in case it contains links to GC-allocated data.

If you take a look inside $ATSHOME/prelude/memory.sats (which depends on
abstract views such as free_gc), then you see could how: by declaring
abstract views and giving types to external C functions. In this case, we
have:

  • abstract view freebyte_ngc_v (N, L), which stands for a linear
    proposition “N bytes of memory are allocated at address L”
  • dataview malloc_v, which handles allocation failure and success; in
    case of success, you get both an at-view and a freebyte_ngc_v – the former
    allows you to write to the pointer, while the latter gives you a capability
    to free the memory
  • function malloc_ngc, implemented externally
  • function free_ngc, implemented externally

Since malloc_ngc and free_ngc are implemented in the GC, they are not very
good examples of using a custom memory allocator, though.
Please provide more details of what you intend to achieve, and I’ll
hopefully be able to give a more specific response.

Cheers,
Artyom Shalkhakov

5

Hi Artyom,

The usage I had in mind is pretty simple: when writing a server-side
extension to Postgresql you have to use palloc() instead of malloc(), and
pfree() instead of free(). Using pfree is to some extent optional: the
memory allocated with pfree() is automatically freed at the end of request
(there are some finer details here - for example, in some cases you have to
switch to another memory storage for data to be more persistent - and it
would be nice if powerful type system of ATS could help to ensure
consistency here - but in the first approximation these problems can be
ignored).

I a case like that I wouldn’t use the GC, I’d just use the linear memory
functions. Wrap the calls to ‘palloc’ and ‘pfree’ such that they work like
the ATS function “ptr_alloc” or similar and use linear types to ensure you
‘pfree’. If you don’t want to call the ‘pfree’ just write a function that
consumes it but does nothing.

http://www.bluishcoder.co.nz

6

Hi Artyom,

The usage I had in mind is pretty simple: when writing a server-side
extension to Postgresql you have to use palloc() instead of malloc(), and
pfree() instead of free(). Using pfree is to some extent optional: the
memory allocated with pfree() is automatically freed at the end of request
(there are some finer details here - for example, in some cases you have to
switch to another memory storage for data to be more persistent - and it
would be nice if powerful type system of ATS could help to ensure
consistency here - but in the first approximation these problems can be
ignored).

In Apache, the idea is very similar, but API is a little bit different: you
use apr_palloc(pool, sizeof(mytype)) instead of malloc(sizeof(mytype)).
There is no pfree in this case.

Can I somehow convince ATS to use this model instead of malloc/free and GC?

Thank you,

 Valentyn.
7

Okay, I will answer this one.

This is extremely easy to do. Unfortunately, it is not properly documented
yet.

First, please use atsopt (instead of atscc) to develop your project.

There are a few memory functions (ats_malloc, ats_free, ats_calloc,
ats_realloc) that
needs to be implemented based on palloc, pfree, etc. That is essentially
all.

–Hongwei