@hayley

you can use one PL, but not one paradigm and not one run-time. At a certain point you had to introduce Actor semantic if you want to manage complex distributed systems, or logical programming if you want to specify complex business rules, and so on. In the end you have many PL paradigms, using more or less the same syntax. It is the Common Lisp approach.

@ramin_hal9001 @rml

@mzan
are you saying that Basically The Good Algorithm Here mostly has one, or a few quite analogous approaches to implementation? But I'm supportive of DSLs and the MOP in common lisp. Or [I forgot the punny name of Kiczalez' implementation of MOP for schemes] for rml and ramin
@hayley @ramin_hal9001 @rml

@screwtape @mzan @hayley @rml for what it's worth, I don't like the idea of using lots of runtimes, it increases complexity, attack surface, opportunities for bugs. But I do agree that you can't just use one. There are times when you really do need an interpreter, and times when you really do need good single-threaded performance without garbage collection, and times when you really do need massive networks of objects with asynchronous object passing, and times when you need good performance measurements and instrumentation, and so on.

That said, I see no reason why all of these different runtimes couldn't be implemented in a version of Lisp that used Rust-like memory management, with the higher level parsing and intermediate language representation implemented in an ordinary garbage collected Lisp.

@ramin_hal9001
I can't see that rust-like memory management could be important. You mean you want a compile time garauntee from the compiler that no consing is happening in this block? I don't think it's that hard to write sbcl code that doesn't cons: But garbage collection is fundamentally good.
@mzan @hayley @rml

@ramin_hal9001 @mzan @hayley @rml

@awkravchuk are you happy with your fine-grained control of memory in sbcl, in terms of being able to write some particularly tight codes without consing? Do you have commentary on that?

@screwtape I love the level of control CL gives in this regard (and SBCL specifcally). I'd say CL is most low-level language from high-level ones 😅​
Although it is not ubiquitous. Just yesterday I've stumbled upon the fact that SBCL would not stack-allocate SAPs (those are 2 machine words wide wrappers around C pointers, which you require to interact with C libraries), even with dynamic-extent declaration.

@ramin_hal9001 @mzan @hayley @rml

@screwtape @mzan @hayley @rml garbage collection is good, except when you want to write operating system kernels, especially for real time applications. I think it is better for the compiler be able to provide some guarantees that memory isn't just being allocated on the heap without being able to prove some programmatic path exists to the end of life of that allocation.

I haven't tried playing with it yet, but I understand that PreScheme which was a subset of Scheme used to compile the Scheme48 compiler, is a non-garbage collected Scheme.

@ramin_hal9001 @screwtape @mzan @hayley @awkravchuk

what do you think about the ephemeron-guardian-weak pairs trifecta for manually managing memory in a garbage collected language? I've just started playing with it in chez, but havent thought of a good test case for it yet.

also how does SBCL memory management work? for some reason I think I recall that much of its tooling for bare bones bit twiddling resides on the FFI side of things, but its quite sophisticated, with FFI not specifically being tied to interfacing with other languages.

@rml

you are lucky: @hayley worked on the SBCL GC, so she can answer.

My 2cents are that, apart the GC, CL gives you a lot of power for optimizing memory management because: with MOP you can customize the memory layout of CLOS objects; if performances are important, you can work on big-chunk of data, stored in arrays, and here CL can give you a lot of options for specifying their format. In both cases, you are reducing the work for the GC, because you have less objects, and you can control (at least partially) how they are stored.

It is a lot of power, that few PL offers.

@ramin_hal9001 @screwtape @awkravchuk

@ramin_hal9001 @screwtape @mzan @rml

> except when you want to write operating system kernels

The Biscuit kernel written in Go suffered a maximum pause of 110µs.

> especially for real time applications

1978 called, wants its real time GC back: https://plover.com/~mjd/misc/hbaker-archive/RealTimeGC.html

> For OS applications, I think it is better for the compiler be able to provide some guarantees that memory isn't just being allocated on the heap without being able to prove some programmatic path exists to the end of life of that allocation.

I don't care.

> PreScheme

...it is in name only, for it is neither Pre- nor Scheme.

> That said, I see no reason why all of these different runtimes couldn't be implemented in a version of Lisp that used Rust-like memory management

And Lisp that would be in name only.

@rml @screwtape @mzan

@hayley

> "1978 called, wants its real time GC back: "

Wow, that is really cool! I had never heard of the Real-Time Garbage Collector before. Is this like an optional feature of SBCL, or do you happen to know if it is in use anywhere in modern languages? Was it used in the Biscuit OS kernel? I will be sure to read that paper. But I am curious why I don't see real-time garbage collectors more widely used, or maybe it is and I am just haven't heard of it before, I am certainly no garbage collection guru.

> "I don't care."

Well, some people (not me) do write operating systems an they do care that their operating system not leak memory or stopping the world to collect garbage. I don't have time to look it up now, but I remember an interview with Linux Torvalds a long time ago about why he insists on using C and not any other high-level language for Linux, and garbage collection and space leaks was one of the things I recall him talking about as a reason why he wouldn't trust other languages. But nowadays, obviously, they are starting to allow Rust into the kernel, and I am sure it is for the very reason that I mentioned, which is that the type checker can provide some guarantees that the kernel will not leak space or stop the world.

> "it is in name only, for it is neither Pre- nor Scheme."

That is debatable. So yes, there are no closures, and it implements type checking and inference, but the rest of the language is similar enough to Scheme that people who read the code can understand it as a Scheme program, and you can reuse part of the pre-Scheme compiler and runtime in the Scheme compiler and runtime as well. But I am really not interested in debates over semantics so I will leave it at that.

@ramin_hal9001 @rml @screwtape @mzan

> Is this like an optional feature of SBCL, or do you happen to know if it is in use anywhere in modern languages?

ZGC and Shenandoah in OpenJDK are similar; SBCL lacks any concurrent collectors. Biscuit uses the Go GC which is another concurrent but non-moving scheme.

> are not stopping the world to collect garbage.

As above.

> But I am really not interested in debates over semantics so I will leave it at that.

Don't start one then, smartass.

@hayley @rml @screwtape @mzan

> "But I am really not interested in debates over semantics so I will leave it at that."

> "Don't start one then, smartass."

I am pretty sure I didn't start a semantic debate. Remember it was you who said of Pre-Scheme and a "Lisp without garbage collection" would be "in name only." You are entitled to your opinion on whether "Pre-Scheme" counts as a Scheme, I say it counts 🙂

But anyway, you are clearly the expert on garbage collection here, and I love it when someone smarter than me disagrees with me because it gives me a chance to challenge my assumptions. (Incidentally, it was when William Byrd, co-author of Minikanren, told me he hated Haskell that I started to question my devotion to System-F.)

Until now I had always thought of garbage collection as something restricted to the realm of higher-level languages, where you want to approximate the behavior of a Turing Machine with an infinitely long tape, and that GC should be left out of lower-level, performance critical stuff (like operating systems) because you shouldn't make any assumptions about infinite memory at that level of abstraction. But you seem to know better: that GC can actually be used in performance critical or real time applications. That is fascinating! I will definitely read the paper you linked my by Henry G. Baker, and thanks for taking the time to tell to me about it.

@ramin_hal9001

> That said, I see no reason why all of these different runtimes couldn't be implemented in a version of Lisp that used Rust-like memory management,

The concept of run-time is wider than a Rust-like "minimum common denominator".

A DBMS with its index types and its query plans is a "run-time". It offers various operations with specific cost-models.

The BEAM with its fine-grained concurrency is a run-time minimizing latency, but with rather limited bandwidth (they are working on a JIT). If you try to call C libraries into BEAM code, you increase bandwidth but you can ruin totally the latency aspects, and you can block services. So, you had to implement the BEAM in Rust, instead of C, but it will be a BEAM run-time, not a Rust run-time.

Linux kernel BPF code is an ad-hoc high-level managed language that can be compiled to efficient code, taking in account also all other submitted BPF fragments submitted by various applications. If you give to this language all the power of Rust, the kernel cannot check and optimize it.

GPU has its specific run-time, and the code must follow a certain paradigm.

The run-time can be affected by the characteristics of the hardware and/or of the problem domain. For the programmer, it is usually a simplified model from which you can derive the approximate cost of operations; the error that are signaled; etc...

We live in a multi-paradigm and multi-runtime world. We cannot escape this.

@screwtape @hayley @rml

@screwtape @hayley @rml

@mzan I had Erlang BEAM specifically in mind when I mentioned that there are "times when you really do need massive networks of objects with asynchronous object passing".

I mentioned features of a few vastly different language runtimes by their characteristics (C, BEAM, Petite Chez), and you mentioned a few others (SQL, Cuda, Linux BPF), and I will say again that I agree with you that we live in a multi-paradigm, multi-runtime world.

But what I meant to say, when I said:

> "I see no reason why all of these different runtimes couldn't be implemented in a version of Lisp that used Rust-like memory management."

was that there are many paradigms and many language runtimes, but since it is possible to implement a runtime in any language at all, it is theoretically possible that all language runtimes be implemented in just one language, for example Rust or Pre-Scheme. It does not even need to have a Rust-like borrow checker, just that when you construct a runtime for a programming language, it is usually better to write the runtime in a language without garbage collection so that you don't force the garbage collection into the language you are implementing (which may not want it), rather you leave it as an optional feature.

@ramin_hal9001 @screwtape @mzan @hayley

part of what I like about scheme is the fact that there are implementations for nearly every niche. and the idea that learning one scheme doesn't mean you'll understand others is wrong I think. if you spend a year in r6/r7rs, you'll be able to dive into others. for that reason I've started doubting whether the usual advice to start with guile is the right advice, its too non-standard.

@rml @ramin_hal9001 @screwtape @mzan@qoto.org @hayley

https://ccrma.stanford.edu/software/snd/snd/s7.html

@rml @ramin_hal9001 @screwtape @hayley

"It exists as just two files, s7.c and s7.h, that want only to disappear into someone else's source tree. There are no libraries, no run-time init files, and no configuration scripts."

@xkummerer @ramin_hal9001 @screwtape @hayley yeah, I there is s7 for Pd now, and the author told me that integrating it was as easy as can be.

@xkummerer @ramin_hal9001 @screwtape @hayley I really need to play with s7, the problem is whenever I do DSP audio stuff these days I reach for Faust, which slaps pretty damn good

@rml @ramin_hal9001 @screwtape @hayley I love faust too

@xkummerer @ramin_hal9001 @screwtape @hayley for anything that doesn't get into spectral synthesis territory, its just so damn good. you can pick up the signal graph of any filter and just implement it, usually in less than 10 lines of code. its one of the best examples of an DSL I can think of.

@xkummerer @ramin_hal9001 @screwtape @hayley and plus it compiles to libraries or extensions for nearly every DSP language or audio DAW. its so productive its insane.

@rml @ramin_hal9001 @screwtape @hayley plus writing a new wrapper is so damn easy

@hayley @ramin_hal9001 @mzan

One Big Language