# Can ATS support LMS style embedded compiler?

**URL:** <https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133>\
**Category:** Google Group\
**Created:** [February 8, 2015, 7:20pm UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133 "2015-02-08T19:20:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![H\_Zhang](https://avatars.discourse-cdn.com/v4/letter/h/b77776/32.png) [@H\_Zhang](https://discourse.ats-lang.org/u/H_Zhang)\
**Post date:** [February 8, 2015, 7:20pm UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/1 "2015-02-08T19:20:23Z")

</div>

LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
[http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
_Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
Regret for High-Level High-Performance Programming_I am still studying it  
but this seems like a very interesting approach to solve the expression  
problem without sacrificing performance. Let me quote from the tutorial page  
[http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):

"Why Generative Programming ?

Because it enables abstraction without regret: the key idea is to write  
very high-level  
and generic programs that generate specialized and extremely fast low-level  
code at runtime.

Programming abstractions that would usually be avoided in  
performance-sensitive code  
(objects, type classes, higher-order functions) can be used during  
generation without  
affecting performance of the generated code.

Why LMS ?

Because it is lightweight and modular: LMS is just a Scala library. With  
types, it distinguishes  
expressions that are evaluated now (type T) vs later (type Rep[T]).

LMS comes with batteries included: it provides many optimizations such as  
common subexpression elimination out of the box, and it goes beyond purely  
generative approaches by  
providing an extensible intermediate representation that can be used to  
implement  
sophisticated domain-specific compiler pipelines.

At the same time LMS is hackable: since it is just a library, all aspects  
can be  
modified or extended to suit a particular purpose."

I wonder if ATS’s very expressive approach can support such a paradigm. For  
a motivational example (motivation for me anyway) see:A SQL Query Compiler  
[http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This approach is used in  
LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very good effect. I think it is  
very attractive if we can mix in relational declarative programming this  
way with imperative programming that can supply large amount of libraries  
and take care of side effects.

---

<div class="post-metadata">

**Author:** ![gmhwxi](https://avatars.discourse-cdn.com/v4/letter/g/67e7ee/32.png) [@gmhwxi](https://discourse.ats-lang.org/u/gmhwxi)\
**Post date:** [February 9, 2015, 12:10am UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/2 "2015-02-09T00:10:54Z")

</div>

> > ATS itself is a code transformer (at compile time) but can it also easily  
> > let users write a domain specific transformer (at run time) on their own?  
> > Can ATS’s dependent type system provide a better tool?

Yes, I would think so.

The type system of ATS makes it highly suitable for dealing with (typeful)  
syntax trees.

Also, I suppose that the compiler for a query language should itself be  
efficient if it is used on-line.On Sun, Feb 8, 2015 at 6:03 PM, Haitao Zhang [zht...@gmail.com](mailto:zht...@gmail.com) wrote:

> Yes that is what I vaguely understood as well and why I asked the question  
> here. However a LMS program is a user defined specializer that generates  
> another program so you can start with a query string and end with a C  
> program (another string). The underlying virtualized Scala is used to  
> ensure type safety. LMS also provides facilities to rewrite the code using  
> the IR. How can one achieve the same with ATS? ATS itself is a code  
> transformer (at compile time) but can it also easily let users write a  
> domain specific transformer (at run time) on their own? Can ATS’s dependent  
> type system provide a better tool?
> 
> On Sun, Feb 8, 2015 at 2:11 PM, gmhwxi [gmh...@gmail.com](mailto:gmh...@gmail.com) wrote:
> 
> > I took a quick look at LMS.
> > 
> > I feel that the template system of ATS is already a kind of LMS.  
> > Instead of using an intermediate representation, ATS uses a small set of  
> > C and then relies on an optimizing C compiler to get efficient object  
> > code.  
> > Compared to LMS, the template-based approach of ATS is more  
> > direct and flexible.
> > 
> > I am not familiar with compiling query languages, though.
> > 
> > On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:
> > 
> > > LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> > > For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> > > [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> > > _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> > > Regret for High-Level High-Performance Programming_I am still studying  
> > > it but this seems like a very interesting approach to solve the expression  
> > > problem without sacrificing performance. Let me quote from the tutorial  
> > > page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> > > 
> > > "Why Generative Programming ?
> > > 
> > > Because it enables abstraction without regret: the key idea is to write  
> > > very high-level  
> > > and generic programs that generate specialized and extremely fast  
> > > low-level code at runtime.
> > > 
> > > Programming abstractions that would usually be avoided in  
> > > performance-sensitive code  
> > > (objects, type classes, higher-order functions) can be used during  
> > > generation without  
> > > affecting performance of the generated code.
> > > 
> > > Why LMS ?
> > > 
> > > Because it is lightweight and modular: LMS is just a Scala library. With  
> > > types, it distinguishes  
> > > expressions that are evaluated now (type T) vs later (type Rep[T]).
> > > 
> > > LMS comes with batteries included: it provides many optimizations such  
> > > as common subexpression elimination out of the box, and it goes beyond  
> > > purely generative approaches by  
> > > providing an extensible intermediate representation that can be used to  
> > > implement  
> > > sophisticated domain-specific compiler pipelines.
> > > 
> > > At the same time LMS is hackable: since it is just a library, all  
> > > aspects can be  
> > > modified or extended to suit a particular purpose."
> > > 
> > > I wonder if ATS’s very expressive approach can support such a paradigm.  
> > > For a motivational example (motivation for me anyway) see:A SQL Query  
> > > Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This  
> > > approach is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very  
> > > good effect. I think it is very attractive if we can mix in relational  
> > > declarative programming this way with imperative programming that can  
> > > supply large amount of libraries and take care of side effects.
> > > 
> > > –  
> > > You received this message because you are subscribed to a topic in the  
> > > Google Groups “ats-lang-users” group.  
> > > To unsubscribe from this topic, visit  
> > > [https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe](https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe).  
> > > To unsubscribe from this group and all its topics, send an email to  
> > > [ats-lang-user...@googlegroups.com](mailto:ats-lang-user...@googlegroups.com).  
> > > To post to this group, send email to [ats-lan...@googlegroups.com](mailto:ats-lan...@googlegroups.com).  
> > > Visit this group at [http://groups.google.com/group/ats-lang-users](http://groups.google.com/group/ats-lang-users).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .
> 
> –  
> You received this message because you are subscribed to the Google Groups  
> “ats-lang-users” group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [ats-lang-user...@googlegroups.com](mailto:ats-lang-user...@googlegroups.com).  
> To post to this group, send email to [ats-lan...@googlegroups.com](mailto:ats-lan...@googlegroups.com).  
> Visit this group at [http://groups.google.com/group/ats-lang-users](http://groups.google.com/group/ats-lang-users).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/ats-lang-users/CAFUyvhqR6P%3DY%3DPh\_VMb3XP\_6OMH10T5RYHnkPXR9Jdhj9fAUcQ%40mail.gmail.com](https://groups.google.com/d/msgid/ats-lang-users/CAFUyvhqR6P%3DY%3DPh_VMb3XP_6OMH10T5RYHnkPXR9Jdhj9fAUcQ%40mail.gmail.com)  
> [https://groups.google.com/d/msgid/ats-lang-users/CAFUyvhqR6P%3DY%3DPh\_VMb3XP\_6OMH10T5RYHnkPXR9Jdhj9fAUcQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/ats-lang-users/CAFUyvhqR6P%3DY%3DPh_VMb3XP_6OMH10T5RYHnkPXR9Jdhj9fAUcQ%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> .

---

<div class="post-metadata">

**Author:** ![H\_Zhang](https://avatars.discourse-cdn.com/v4/letter/h/b77776/32.png) [@H\_Zhang](https://discourse.ats-lang.org/u/H_Zhang)\
**Post date:** [February 9, 2015, 7:53am UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/3 "2015-02-09T07:53:46Z")

</div>

This paper [http://theory.stanford.edu/~aiken/publications/papers/pldi11b.pdf](http://theory.stanford.edu/~aiken/publications/papers/pldi11b.pdf)  
is shorter and more succinct.On Sun, Feb 8, 2015 at 11:40 PM, Haitao Zhang [zht...@gmail.com](mailto:zht...@gmail.com) wrote:

> RelC as described in this dissertation  
> [http://theory.stanford.edu/~aiken/publications/theses/hawkins.pdf](http://theory.stanford.edu/~aiken/publications/theses/hawkins.pdf) is a  
> good approximation of what I would like to see in synthesizing code from  
> relational descriptions. The dissertation does not give implementation  
> details.
> 
> On Sun, Feb 8, 2015 at 2:11 PM, gmhwxi [gmh...@gmail.com](mailto:gmh...@gmail.com) wrote:
> 
> > I took a quick look at LMS.
> > 
> > I feel that the template system of ATS is already a kind of LMS.  
> > Instead of using an intermediate representation, ATS uses a small set of  
> > C and then relies on an optimizing C compiler to get efficient object  
> > code.  
> > Compared to LMS, the template-based approach of ATS is more  
> > direct and flexible.
> > 
> > I am not familiar with compiling query languages, though.
> > 
> > On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:
> > 
> > > LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> > > For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> > > [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> > > _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> > > Regret for High-Level High-Performance Programming_I am still studying  
> > > it but this seems like a very interesting approach to solve the expression  
> > > problem without sacrificing performance. Let me quote from the tutorial  
> > > page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> > > 
> > > "Why Generative Programming ?
> > > 
> > > Because it enables abstraction without regret: the key idea is to write  
> > > very high-level  
> > > and generic programs that generate specialized and extremely fast  
> > > low-level code at runtime.
> > > 
> > > Programming abstractions that would usually be avoided in  
> > > performance-sensitive code  
> > > (objects, type classes, higher-order functions) can be used during  
> > > generation without  
> > > affecting performance of the generated code.
> > > 
> > > Why LMS ?
> > > 
> > > Because it is lightweight and modular: LMS is just a Scala library. With  
> > > types, it distinguishes  
> > > expressions that are evaluated now (type T) vs later (type Rep[T]).
> > > 
> > > LMS comes with batteries included: it provides many optimizations such  
> > > as common subexpression elimination out of the box, and it goes beyond  
> > > purely generative approaches by  
> > > providing an extensible intermediate representation that can be used to  
> > > implement  
> > > sophisticated domain-specific compiler pipelines.
> > > 
> > > At the same time LMS is hackable: since it is just a library, all  
> > > aspects can be  
> > > modified or extended to suit a particular purpose."
> > > 
> > > I wonder if ATS’s very expressive approach can support such a paradigm.  
> > > For a motivational example (motivation for me anyway) see:A SQL Query  
> > > Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This  
> > > approach is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very  
> > > good effect. I think it is very attractive if we can mix in relational  
> > > declarative programming this way with imperative programming that can  
> > > supply large amount of libraries and take care of side effects.
> > > 
> > > –  
> > > You received this message because you are subscribed to a topic in the  
> > > Google Groups “ats-lang-users” group.  
> > > To unsubscribe from this topic, visit  
> > > [https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe](https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe).  
> > > To unsubscribe from this group and all its topics, send an email to  
> > > [ats-lang-user...@googlegroups.com](mailto:ats-lang-user...@googlegroups.com).  
> > > To post to this group, send email to [ats-lan...@googlegroups.com](mailto:ats-lan...@googlegroups.com).  
> > > Visit this group at [http://groups.google.com/group/ats-lang-users](http://groups.google.com/group/ats-lang-users).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .

---

<div class="post-metadata">

**Author:** ![H\_Zhang](https://avatars.discourse-cdn.com/v4/letter/h/b77776/32.png) [@H\_Zhang](https://discourse.ats-lang.org/u/H_Zhang)\
**Post date:** [February 9, 2015, 7:40am UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/4 "2015-02-09T07:40:01Z")

</div>

RelC as described in this dissertation  
[http://theory.stanford.edu/~aiken/publications/theses/hawkins.pdf](http://theory.stanford.edu/~aiken/publications/theses/hawkins.pdf) is a good  
approximation of what I would like to see in synthesizing code from  
relational descriptions. The dissertation does not give implementation  
details.On Sun, Feb 8, 2015 at 2:11 PM, gmhwxi [gmh...@gmail.com](mailto:gmh...@gmail.com) wrote:

> I took a quick look at LMS.
> 
> I feel that the template system of ATS is already a kind of LMS.  
> Instead of using an intermediate representation, ATS uses a small set of  
> C and then relies on an optimizing C compiler to get efficient object code.  
> Compared to LMS, the template-based approach of ATS is more  
> direct and flexible.
> 
> I am not familiar with compiling query languages, though.
> 
> On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:
> 
> > LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> > For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> > [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> > _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> > Regret for High-Level High-Performance Programming_I am still studying  
> > it but this seems like a very interesting approach to solve the expression  
> > problem without sacrificing performance. Let me quote from the tutorial  
> > page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> > 
> > "Why Generative Programming ?
> > 
> > Because it enables abstraction without regret: the key idea is to write  
> > very high-level  
> > and generic programs that generate specialized and extremely fast  
> > low-level code at runtime.
> > 
> > Programming abstractions that would usually be avoided in  
> > performance-sensitive code  
> > (objects, type classes, higher-order functions) can be used during  
> > generation without  
> > affecting performance of the generated code.
> > 
> > Why LMS ?
> > 
> > Because it is lightweight and modular: LMS is just a Scala library. With  
> > types, it distinguishes  
> > expressions that are evaluated now (type T) vs later (type Rep[T]).
> > 
> > LMS comes with batteries included: it provides many optimizations such as  
> > common subexpression elimination out of the box, and it goes beyond purely  
> > generative approaches by  
> > providing an extensible intermediate representation that can be used to  
> > implement  
> > sophisticated domain-specific compiler pipelines.
> > 
> > At the same time LMS is hackable: since it is just a library, all aspects  
> > can be  
> > modified or extended to suit a particular purpose."
> > 
> > I wonder if ATS’s very expressive approach can support such a paradigm.  
> > For a motivational example (motivation for me anyway) see:A SQL Query  
> > Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This  
> > approach is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very good  
> > effect. I think it is very attractive if we can mix in relational  
> > declarative programming this way with imperative programming that can  
> > supply large amount of libraries and take care of side effects.
> > 
> > –  
> > You received this message because you are subscribed to a topic in the  
> > Google Groups “ats-lang-users” group.  
> > To unsubscribe from this topic, visit  
> > [https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe](https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe).  
> > To unsubscribe from this group and all its topics, send an email to  
> > [ats-lang-user...@googlegroups.com](mailto:ats-lang-user...@googlegroups.com).  
> > To post to this group, send email to [ats-lan...@googlegroups.com](mailto:ats-lan...@googlegroups.com).  
> > Visit this group at [http://groups.google.com/group/ats-lang-users](http://groups.google.com/group/ats-lang-users).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .

---

<div class="post-metadata">

**Author:** ![gmhwxi](https://avatars.discourse-cdn.com/v4/letter/g/67e7ee/32.png) [@gmhwxi](https://discourse.ats-lang.org/u/gmhwxi)\
**Post date:** [February 8, 2015, 10:11pm UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/5 "2015-02-08T22:11:55Z")

</div>

I took a quick look at LMS.

I feel that the template system of ATS is already a kind of LMS.  
Instead of using an intermediate representation, ATS uses a small set of  
C and then relies on an optimizing C compiler to get efficient object code.  
Compared to LMS, the template-based approach of ATS is more  
direct and flexible.

I am not familiar with compiling query languages, though.On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:

> LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> Regret for High-Level High-Performance Programming_I am still studying it  
> but this seems like a very interesting approach to solve the expression  
> problem without sacrificing performance. Let me quote from the tutorial  
> page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> 
> "Why Generative Programming ?
> 
> Because it enables abstraction without regret: the key idea is to write  
> very high-level  
> and generic programs that generate specialized and extremely fast  
> low-level code at runtime.
> 
> Programming abstractions that would usually be avoided in  
> performance-sensitive code  
> (objects, type classes, higher-order functions) can be used during  
> generation without  
> affecting performance of the generated code.
> 
> Why LMS ?
> 
> Because it is lightweight and modular: LMS is just a Scala library. With  
> types, it distinguishes  
> expressions that are evaluated now (type T) vs later (type Rep[T]).
> 
> LMS comes with batteries included: it provides many optimizations such as  
> common subexpression elimination out of the box, and it goes beyond purely  
> generative approaches by  
> providing an extensible intermediate representation that can be used to  
> implement  
> sophisticated domain-specific compiler pipelines.
> 
> At the same time LMS is hackable: since it is just a library, all aspects  
> can be  
> modified or extended to suit a particular purpose."
> 
> I wonder if ATS’s very expressive approach can support such a paradigm.  
> For a motivational example (motivation for me anyway) see:A SQL Query  
> Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This approach  
> is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very good effect. I  
> think it is very attractive if we can mix in relational declarative  
> programming this way with imperative programming that can supply large  
> amount of libraries and take care of side effects.

---

<div class="post-metadata">

**Author:** ![gmhwxi](https://avatars.discourse-cdn.com/v4/letter/g/67e7ee/32.png) [@gmhwxi](https://discourse.ats-lang.org/u/gmhwxi)\
**Post date:** [February 9, 2015, 12:32am UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/6 "2015-02-09T00:32:07Z")

</div>

To put on the hat of a devil’s advocate:

Using LMS to generate efficient C code is incredibly hard; so LMS will  
become more and more  
complex; so people will find out later that writing efficient C code  
directly is probably more cost-effective.On Sunday, February 8, 2015 at 6:20:54 PM UTC-5, H Zhang wrote:

> Also I think one of the argument for LMS is that once code is transformed  
> into low level code (C for example) many optimization opportunities are  
> lost esp if one starts with high level representations (high order  
> functions, polymorphism e.g.), so their motto is “abstraction without  
> regret”. I don’t know how well that can be achieved but it certainly is a  
> very admirable goal.
> 
> On Sun, Feb 8, 2015 at 2:11 PM, gmhwxi \<[gmh...@gmail.com](mailto:gmh...@gmail.com) \<javascript:\>\> wrote:
> 
> > I took a quick look at LMS.
> > 
> > I feel that the template system of ATS is already a kind of LMS.  
> > Instead of using an intermediate representation, ATS uses a small set of  
> > C and then relies on an optimizing C compiler to get efficient object  
> > code.  
> > Compared to LMS, the template-based approach of ATS is more  
> > direct and flexible.
> > 
> > I am not familiar with compiling query languages, though.
> > 
> > On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:
> > 
> > > LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> > > For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> > > [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> > > _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> > > Regret for High-Level High-Performance Programming_I am still studying  
> > > it but this seems like a very interesting approach to solve the expression  
> > > problem without sacrificing performance. Let me quote from the tutorial  
> > > page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> > > 
> > > "Why Generative Programming ?
> > > 
> > > Because it enables abstraction without regret: the key idea is to write  
> > > very high-level  
> > > and generic programs that generate specialized and extremely fast  
> > > low-level code at runtime.
> > > 
> > > Programming abstractions that would usually be avoided in  
> > > performance-sensitive code  
> > > (objects, type classes, higher-order functions) can be used during  
> > > generation without  
> > > affecting performance of the generated code.
> > > 
> > > Why LMS ?
> > > 
> > > Because it is lightweight and modular: LMS is just a Scala library. With  
> > > types, it distinguishes  
> > > expressions that are evaluated now (type T) vs later (type Rep[T]).
> > > 
> > > LMS comes with batteries included: it provides many optimizations such  
> > > as common subexpression elimination out of the box, and it goes beyond  
> > > purely generative approaches by  
> > > providing an extensible intermediate representation that can be used to  
> > > implement  
> > > sophisticated domain-specific compiler pipelines.
> > > 
> > > At the same time LMS is hackable: since it is just a library, all  
> > > aspects can be  
> > > modified or extended to suit a particular purpose."
> > > 
> > > I wonder if ATS’s very expressive approach can support such a paradigm.  
> > > For a motivational example (motivation for me anyway) see:A SQL Query  
> > > Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This  
> > > approach is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very  
> > > good effect. I think it is very attractive if we can mix in relational  
> > > declarative programming this way with imperative programming that can  
> > > supply large amount of libraries and take care of side effects.
> > > 
> > > –  
> > > You received this message because you are subscribed to a topic in the  
> > > Google Groups “ats-lang-users” group.  
> > > To unsubscribe from this topic, visit  
> > > [https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe](https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe).  
> > > To unsubscribe from this group and all its topics, send an email to  
> > > [ats-lang...@googlegroups.com](mailto:ats-lang...@googlegroups.com) \<javascript:\>.  
> > > To post to this group, send email to [ats-l...@googlegroups.com](mailto:ats-l...@googlegroups.com)  
> > > \<javascript:\>.  
> > > Visit this group at [http://groups.google.com/group/ats-lang-users](http://groups.google.com/group/ats-lang-users).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .

---

<div class="post-metadata">

**Author:** ![H\_Zhang](https://avatars.discourse-cdn.com/v4/letter/h/b77776/32.png) [@H\_Zhang](https://discourse.ats-lang.org/u/H_Zhang)\
**Post date:** [February 8, 2015, 11:20pm UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/7 "2015-02-08T23:20:54Z")

</div>

Also I think one of the argument for LMS is that once code is transformed  
into low level code (C for example) many optimization opportunities are  
lost esp if one starts with high level representations (high order  
functions, polymorphism e.g.), so their motto is “abstraction without  
regret”. I don’t know how well that can be achieved but it certainly is a  
very admirable goal.On Sun, Feb 8, 2015 at 2:11 PM, gmhwxi [gmh...@gmail.com](mailto:gmh...@gmail.com) wrote:

> I took a quick look at LMS.
> 
> I feel that the template system of ATS is already a kind of LMS.  
> Instead of using an intermediate representation, ATS uses a small set of  
> C and then relies on an optimizing C compiler to get efficient object code.  
> Compared to LMS, the template-based approach of ATS is more  
> direct and flexible.
> 
> I am not familiar with compiling query languages, though.
> 
> On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:
> 
> > LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> > For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> > [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> > _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> > Regret for High-Level High-Performance Programming_I am still studying  
> > it but this seems like a very interesting approach to solve the expression  
> > problem without sacrificing performance. Let me quote from the tutorial  
> > page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> > 
> > "Why Generative Programming ?
> > 
> > Because it enables abstraction without regret: the key idea is to write  
> > very high-level  
> > and generic programs that generate specialized and extremely fast  
> > low-level code at runtime.
> > 
> > Programming abstractions that would usually be avoided in  
> > performance-sensitive code  
> > (objects, type classes, higher-order functions) can be used during  
> > generation without  
> > affecting performance of the generated code.
> > 
> > Why LMS ?
> > 
> > Because it is lightweight and modular: LMS is just a Scala library. With  
> > types, it distinguishes  
> > expressions that are evaluated now (type T) vs later (type Rep[T]).
> > 
> > LMS comes with batteries included: it provides many optimizations such as  
> > common subexpression elimination out of the box, and it goes beyond purely  
> > generative approaches by  
> > providing an extensible intermediate representation that can be used to  
> > implement  
> > sophisticated domain-specific compiler pipelines.
> > 
> > At the same time LMS is hackable: since it is just a library, all aspects  
> > can be  
> > modified or extended to suit a particular purpose."
> > 
> > I wonder if ATS’s very expressive approach can support such a paradigm.  
> > For a motivational example (motivation for me anyway) see:A SQL Query  
> > Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This  
> > approach is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very good  
> > effect. I think it is very attractive if we can mix in relational  
> > declarative programming this way with imperative programming that can  
> > supply large amount of libraries and take care of side effects.
> > 
> > –  
> > You received this message because you are subscribed to a topic in the  
> > Google Groups “ats-lang-users” group.  
> > To unsubscribe from this topic, visit  
> > [https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe](https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe).  
> > To unsubscribe from this group and all its topics, send an email to  
> > [ats-lang-user...@googlegroups.com](mailto:ats-lang-user...@googlegroups.com).  
> > To post to this group, send email to [ats-lan...@googlegroups.com](mailto:ats-lan...@googlegroups.com).  
> > Visit this group at [http://groups.google.com/group/ats-lang-users](http://groups.google.com/group/ats-lang-users).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .

---

<div class="post-metadata">

**Author:** ![H\_Zhang](https://avatars.discourse-cdn.com/v4/letter/h/b77776/32.png) [@H\_Zhang](https://discourse.ats-lang.org/u/H_Zhang)\
**Post date:** [February 8, 2015, 11:03pm UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/8 "2015-02-08T23:03:59Z")

</div>

Yes that is what I vaguely understood as well and why I asked the question  
here. However a LMS program is a user defined specializer that generates  
another program so you can start with a query string and end with a C  
program (another string). The underlying virtualized Scala is used to  
ensure type safety. LMS also provides facilities to rewrite the code using  
the IR. How can one achieve the same with ATS? ATS itself is a code  
transformer (at compile time) but can it also easily let users write a  
domain specific transformer (at run time) on their own? Can ATS’s dependent  
type system provide a better tool?On Sun, Feb 8, 2015 at 2:11 PM, gmhwxi [gmh...@gmail.com](mailto:gmh...@gmail.com) wrote:

> I took a quick look at LMS.
> 
> I feel that the template system of ATS is already a kind of LMS.  
> Instead of using an intermediate representation, ATS uses a small set of  
> C and then relies on an optimizing C compiler to get efficient object code.  
> Compared to LMS, the template-based approach of ATS is more  
> direct and flexible.
> 
> I am not familiar with compiling query languages, though.
> 
> On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:
> 
> > LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> > For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> > [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> > _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> > Regret for High-Level High-Performance Programming_I am still studying  
> > it but this seems like a very interesting approach to solve the expression  
> > problem without sacrificing performance. Let me quote from the tutorial  
> > page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> > 
> > "Why Generative Programming ?
> > 
> > Because it enables abstraction without regret: the key idea is to write  
> > very high-level  
> > and generic programs that generate specialized and extremely fast  
> > low-level code at runtime.
> > 
> > Programming abstractions that would usually be avoided in  
> > performance-sensitive code  
> > (objects, type classes, higher-order functions) can be used during  
> > generation without  
> > affecting performance of the generated code.
> > 
> > Why LMS ?
> > 
> > Because it is lightweight and modular: LMS is just a Scala library. With  
> > types, it distinguishes  
> > expressions that are evaluated now (type T) vs later (type Rep[T]).
> > 
> > LMS comes with batteries included: it provides many optimizations such as  
> > common subexpression elimination out of the box, and it goes beyond purely  
> > generative approaches by  
> > providing an extensible intermediate representation that can be used to  
> > implement  
> > sophisticated domain-specific compiler pipelines.
> > 
> > At the same time LMS is hackable: since it is just a library, all aspects  
> > can be  
> > modified or extended to suit a particular purpose."
> > 
> > I wonder if ATS’s very expressive approach can support such a paradigm.  
> > For a motivational example (motivation for me anyway) see:A SQL Query  
> > Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This  
> > approach is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very good  
> > effect. I think it is very attractive if we can mix in relational  
> > declarative programming this way with imperative programming that can  
> > supply large amount of libraries and take care of side effects.
> > 
> > –  
> > You received this message because you are subscribed to a topic in the  
> > Google Groups “ats-lang-users” group.  
> > To unsubscribe from this topic, visit  
> > [https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe](https://groups.google.com/d/topic/ats-lang-users/vgjuDIMcXYI/unsubscribe).  
> > To unsubscribe from this group and all its topics, send an email to  
> > [ats-lang-user...@googlegroups.com](mailto:ats-lang-user...@googlegroups.com).  
> > To post to this group, send email to [ats-lan...@googlegroups.com](mailto:ats-lan...@googlegroups.com).  
> > Visit this group at [http://groups.google.com/group/ats-lang-users](http://groups.google.com/group/ats-lang-users).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/ats-lang-users/b3b30b9b-d081-4638-8a52-d62825990526%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .

---

<div class="post-metadata">

**Author:** ![gmhwxi](https://avatars.discourse-cdn.com/v4/letter/g/67e7ee/32.png) [@gmhwxi](https://discourse.ats-lang.org/u/gmhwxi)\
**Post date:** [February 8, 2015, 10:27pm UTC](https://discourse.ats-lang.org/t/can-ats-support-lms-style-embedded-compiler/133/9 "2015-02-08T22:27:04Z")

</div>

Please compare the following example with the one at  
[http://scala-lms.github.io](http://scala-lms.github.io):

[https://github.com/githwxi/ATS-Postiats/blob/master/doc/EXAMPLE/ATS-QA-LIST/qa-list-2015-02-08.datsOn](https://github.com/githwxi/ATS-Postiats/blob/master/doc/EXAMPLE/ATS-QA-LIST/qa-list-2015-02-08.datsOn) Sunday, February 8, 2015 at 5:11:56 PM UTC-5, gmhwxi wrote:

> I took a quick look at LMS.
> 
> I feel that the template system of ATS is already a kind of LMS.  
> Instead of using an intermediate representation, ATS uses a small set of  
> C and then relies on an optimizing C compiler to get efficient object code.  
> Compared to LMS, the template-based approach of ATS is more  
> direct and flexible.
> 
> I am not familiar with compiling query languages, though.
> 
> On Sunday, February 8, 2015 at 2:20:23 PM UTC-5, H Zhang wrote:
> 
> > LMS stands for Lightweight Modular Staging [http://scala-lms.github.io/](http://scala-lms.github.io/).  
> > For a definitive guide see Tiark Rompf’s Ph.D. dissertation  
> > [http://infoscience.epfl.ch/record/180642/files/EPFL\_TH5456.pdf](http://infoscience.epfl.ch/record/180642/files/EPFL_TH5456.pdf):  
> > _Lightweight Modular Staging and Embedded Compilers: Abstraction Without  
> > Regret for High-Level High-Performance Programming_I am still studying  
> > it but this seems like a very interesting approach to solve the expression  
> > problem without sacrificing performance. Let me quote from the tutorial  
> > page [http://scala-lms.github.io/tutorials/index.html](http://scala-lms.github.io/tutorials/index.html):
> > 
> > "Why Generative Programming ?
> > 
> > Because it enables abstraction without regret: the key idea is to write  
> > very high-level  
> > and generic programs that generate specialized and extremely fast  
> > low-level code at runtime.
> > 
> > Programming abstractions that would usually be avoided in  
> > performance-sensitive code  
> > (objects, type classes, higher-order functions) can be used during  
> > generation without  
> > affecting performance of the generated code.
> > 
> > Why LMS ?
> > 
> > Because it is lightweight and modular: LMS is just a Scala library. With  
> > types, it distinguishes  
> > expressions that are evaluated now (type T) vs later (type Rep[T]).
> > 
> > LMS comes with batteries included: it provides many optimizations such as  
> > common subexpression elimination out of the box, and it goes beyond purely  
> > generative approaches by  
> > providing an extensible intermediate representation that can be used to  
> > implement  
> > sophisticated domain-specific compiler pipelines.
> > 
> > At the same time LMS is hackable: since it is just a library, all aspects  
> > can be  
> > modified or extended to suit a particular purpose."
> > 
> > I wonder if ATS’s very expressive approach can support such a paradigm.  
> > For a motivational example (motivation for me anyway) see:A SQL Query  
> > Compiler [http://scala-lms.github.io/tutorials/query.html](http://scala-lms.github.io/tutorials/query.html). This  
> > approach is used in LegoBase [http://data.epfl.ch/legobase](http://data.epfl.ch/legobase) to very good  
> > effect. I think it is very attractive if we can mix in relational  
> > declarative programming this way with imperative programming that can  
> > supply large amount of libraries and take care of side effects.