OT: Dynamic-CC, GC, Dynamic Types, and Prototype OO in C - C

for my own dynamic C compiler effort, I have been specifying, and have
already largely implemented, a mass of library functionality offering
Garbage Collection (conservative), Dynamic Typing (through the "magically
typed void pointer" approach), and a Prototype Object System (similar to
that of Self or JavaScript).

for general context (not intended as SPAM, but may come off this way I
guess...):
this is a C compiler which serves to behave more like a traditional VM or
interpreter, namely in that it is capable of compiling and linking code into
the running app at runtime. unlike a traditional VM, I am using C as the
primary language (both for writing the implementation and for the scripts),
and this VM is capable of linking against and directly using existing
statically compiled/linked libraries (for example, stdio, OpenGL, the Win32
API, ...).

the goal is in part to have, in C, many of the capabilities traditionally
available to languages such as LISP, Smalltalk, or Python, while still
retaining the capabilities and raw performance of C (and hopefully
eventually C++ as well, but a C++ compiler looks like a horrid beast to
implement at present).

at present, 32 bit windows is my primary development target.
most of this is being developed under the LGPL.

at present, the compiler still needs a little more work (buggy, incomplete,
and non-standards-conformant in some places).

well, LLVM is another project people can look into (for anyone that wants
this kind of functionality, this project is a lot more usable at present).
my effort is not intended to preclude the efforts of the LLVM folks, just
doing my own thing mostly...


as for the GC, Dynamic Types, and Prototype OO in C:

in all this, the C language itself is left unaffected (this is library
functionality, and does not involve use of custom compiler extensions). as
such, it will not be the "nicest" possible option, but should be workable
(basically, since I am stalling out on my ideas for implementing a scripting
language based on my compiler core, I decided instead to implement all this
as library functionality and stick to more or less "ordinary" C for the
implementation/interface).

neither the spec nor the library are "complete" at this point (I may specify
a lot more than this, and at present have only implemented the GC and
dynamic type system).

I may also include the following other features in this framework:
exceptions; multithreading; file-management/VFS, ...

to a large degree, this is being derived from existing code of mine,
however, in the process I am making fairly extensive modification to the
code in which I am reusing (basically, I have about a 300 kloc existing
codebase that I am mining for pieces in implementing all this). if taken
"all the way", this will also include GL wrapping, a GL based GUI, 3D engine
code, ... however, all this is more likely to undergo much more minor
modification.

basically, in all this, the dynamic C compiler is the core (however, it will
not be a requirement for using any of this functionality).


it is possible, however, that some people may find some of this
"interesting".


a few of the lib specs:
----

GC Basics:
This will be a conservative GC.
Pointers may be freely stored on the stack, in global variables, or in other
garbage-collected objects. The pointer may point anywhere within the
allocated object.

Caution needs to be excersized when mixing GC'ed data with non-GC'ed data
(such as memory gained from malloc, mmap, ...). Likewise when "mutilating"
pointers. In these cases, it is no longer required that the GC be able to
correctly locate or identify these pointers (resulting in 'accidental'
collection).


GC API:
void *gcalloc(size_t sz);
void *gctalloc(char *type, size_t sz);

void *gcatomic(size_t sz);
void *gctatomic(char *type, size_t sz);

void *gcrealloc(void *p, size_t sz);
void *gcfree(void *p);
char *gcgettype(void *p);
int gctypep(void *p, char *ty);

int gcsetfinal(char *ty, void (*fcn)(void *p));

Uncertain Features:
void *gctallocu(char *type, size_t sz); //unaligned alloc
void *gcmalloc(size_t sz); //alloc initially-locked object
void gclock(void *p); //prevent collection
void gcunlock(void *p); //allow collection


gcalloc():
Allocates an object on the garbage-collected heap.
The type of such a value will be viewed as being NULL.

gctalloc():
Allocates a typed object on the GC heap.
The type is a string whose structure is not specified by the GC. For custom
types, effort should be used to ensure that the string does not clash with a
builtin type or one provided by another library.

The creation of a new type may consist simply of allocating an object of the
particular type.

gcatomic()/gctatomic():
Allocates an 'atomic' object on the garbage collected heap.
An atomic object will differ from a normal object primarily in that the
contents of an atomic object will not be scanned for references.
As a result, it will be more suited for safely handling buffers and other
types known not to contain memory references.

gcrealloc():
Reallocate an object to a larger or smaller size.
The type will be be retained from the parent object.

gcfree():
Forcibly free an object. This should only be done when it is known that this
object will no longer be accessed.

gcgettype():
Return the type associated with an object.
If the pointer is not part of the GC's heap, or was gained from gcalloc,
then the return value is NULL.

gctypep():
Returns a nonzero value if the type of p is the same as that given in ty.
This determination will be based on string equality.

gcsetfinal():
Set the finalizer called for a specific type.
This function will be called while freeing objects of this type.


gcmalloc():
If included, would create objects by default resembling those from malloc
(assuming manual memory management, ...).

However, unlike malloc(), the memory returned by gcmalloc() may be searched
for pointers.


----

Dynamic Type System:

GC type names beginning with '_' will be reserved for the implementation.

General Conventions:
void *dy<type>(...);
Will create an object of the type.

int dy<type>p(void *p);
Will return a nonzero value if p is the correct type.

<type> dy<type>v(void *p);
Will get the value for a specific type.

char *dygettype(void *p);
int dytypep(void *p, char *ty);
Will get or check the type of an object.
These will exist as there is no strict requirement that some of these
builtin types will have the same GC type (or even necessarily be on-heap).
In these cases, this function may need to do additional work in order to
locate the correct type string.

If only a single type needs to be checked, the provided specialized
predicate functions will be viewed as the preferred means.


Note:
For Complex Types (such as arrays, conses, and hashes), it is acceptable to
use any kind of pointer (it does not have to be from this type system, or
necessarily even the GC). However, that is the intent that this be used
mostly for dynamic types and GC'ed data, and other GC restrictions apply.


Primitive Types: int; long; long long; float; double; fcomplex; dcomplex.
Basic Types: string; wstring; symbol; keyword.
Complex Types: array; cons; hash.

....

so:
void *dyint(int v);
void *dylong(long v);
void *dylonglong(long long v);
void *dyfloat(float v);
void *dydouble(double v);
void *dyfcomplex(float _Complex v);
void *dydcomplex(double _Complex v);

char *dystring(char *s);
char *dysymbol(char *s);
char *dykeyword(char *s);
wchar_t *dywstring(wchar_t *s);

....

int dyintp(void *p);
int dylongp(void *p);
int dylonglongp(void *p);

....

int dyintv(void *p);
long dylongv(void *p);
long long dylonglongv(void *p);

....


Integer Type Disjointness
It will not be required that integer types be disjoint.
As a result, a value created with dyint() may still return a nonzero value
with dylonglongp().

Likewise, a value created by dylonglong() may be nonzero with dyintp() in
the case where dylonglong() was called with a value sufficiently small to
fit into an integer.

As such, these predicates will mostly serve to indicate range requirements,
rather than a strict type.

Float and Double
These will, however, be regarded as disjoint types.

Real will exist as a psuedo type, which will be true for values decodable as
a double.

int dyrealp(void *p);
double dyrealv(void *p);


Arithmetic
Functions will be provided for arithmetic on dynamic types.

void *dyadd(void *a, void *b);
void *dysub(void *a, void *b);
void *dymul(void *a, void *b);
void *dydiv(void *a, void *b);
void *dymod(void *a, void *b);
void *dyand(void *a, void *b);
void *dyor(void *a, void *b);
void *dyxor(void *a, void *b);

void *dyneg(void *a);
void *dynot(void *a);

int dyeqp(void *a, void *b);
int dyneqp(void *a, void *b);
int dygtp(void *a, void *b);
int dyltp(void *a, void *b);
int dygep(void *a, void *b);
int dylep(void *a, void *b);

int dytruep(void *a);
int dyfalsep(void *a);


String:
Will behave much the same as a GC'ed dynamicly typed strdup.
Additionally, string merging may be performed, so the result should be
regarded as immutable.

Symbol and Keyword:
These will be 2 different types, but each has essentially the same
semantics.

Beyond the different types, they will have another important property:
Any 2 calls to the same function with the same input string produces the
same output (allowing '==' to be used for equality checks).


Array:

void *dyarray(int cnt);
Create an array with initially cnt pointers.

void *dyarrayidx(void *p, int idx);
void dyarraysetidx(void *p, int idx, void *q);
Get and Set array index.

void **dyarrayv(void *p);
Get a 'void **' array representing the contents of the array.


Cons:

void *dycons(void *car, void *cdr);
void *dycar(void *p);
void *dycdr(void *p);

void *dycaar(void *p);
void *dycadr(void *p);
void *dycdar(void *p);
void *dycddr(void *p);
(up to 4 levels).
....

void *dylist1(void *a);
void *dylist2(void *a, void *b);
void *dylist3(void *a, void *b, void *c);
void *dylist4(void *a, void *b, void *c, void *d);
void *dylist5(void *a, void *b, void *c, void *d, void *e);
void *dylist6(void *a, void *b, void *c, void *d, void *e, void *f);
(up to 10).
....


Hash:

void *dyhash(int cnt);
Create a hash with initially cnt slots in the table. Calling this with '0'
will specify that the implementation use a default value.

void *dyhashget(void *p, char *s);
void *dyhashset(void *p, char *s, void *q);
Get or Set the value associated with a given string index.
Set will return the previous value (or NULL).

The hash may be resized if it is full.


----

Dynamically Typed Object Orientation

Purpose:
An dynamic object system is provided.
This will provide a general prototype-based OO setup.
The purpose will not to be "high performance", or to replace more
traditional object systems or limit custom types/data, however, a goal will
be that performance remain "acceptable" for many tasks.

Objects will use a Prototype Object System.

In this system, each object itself is viewed primarily as a bag of slots,
with additional behaviors being gained by 'delegation'. In this system, a
failure to locate a slot in a given object causes the slot to be looked up.
Within this system, multiple delegates are allowed per-object (however,
"parent" is the default). Additionally, the delegation graph will be allowed
to be cyclic.

Some calls may return a special value 'UNDEFINED', which will signal an
error condition (unlike NULL, which simply means 'no value'). It will be
considered invalid to use this value otherwise (this value is only returned,
in error conditions, and is not to be passed around or stored). Later this
value may also be used as a means of raising exceptions.

When calling a method, a special method 'doesNotUnderstand' may be called
prior to returning with an error condition. The 'doesNotUnderstand' method
will be called with the same arguments as the original method call, but the
args list will be prefixed with the method name.


Conventions:
DyOO will use a variant camelCase naming scheme.


DyOO will have 2 major interfaces:

Generating code to interface with existing dynamic types. This will be done
by registering certain callbacks with the types. This will be intended
mostly for types that are implemented in plain C (as structs, ...), but for
whatever reason it is useful to be able to them access dynamically.

A full prototype system, where methods are objects in themselves.

For the former interface, these functions are provided.
void dyTypeGetSlot(char *ty, dyt (*fcn)(dyt obj, dyt sym));
void dyTypeSetSlot(char *ty, dyt (*fcn)(dyt obj, dyt sym, dyt val));
void dyTypeNextSlot(char *ty, dyt (*fcn)(dyt obj, dyt sym));
void dyTypeCallMethod(char *ty, dyt (*fcn)(dyt obj, dyt sym, dyt *args, int
nargs));
void dyTypeApply(char *ty, dyt (*fcn)(dyt obj, dyt *args, int nargs));


For the latter interface:
dyt dyFunc(dyt (*fcn)(dyt *args, int nargs));
dyt dyMethod(dyt (*fcn)(dyt self, dyt *args, int nargs));
Create a function or method.
The primary difference between them will be the existence of the 'self'
argument.

Additional forms will be provided for functions with a fixed number of args:
dyt dyFunc0(dyt (*fcn)());
dyt dyFunc1(dyt (*fcn)(dyt));
dyt dyFunc2(dyt (*fcn)(dyt, dyt));
dyt dyFunc3(dyt (*fcn)(dyt, dyt, dyt));
dyt dyFunc4(dyt (*fcn)(dyt, dyt, dyt, dyt));
dyt dyFunc5(dyt (*fcn)(dyt, dyt, dyt, dyt, dyt));
dyt dyFunc6(dyt (*fcn)(dyt, dyt, dyt, dyt, dyt, dyt));
dyt dyFunc7(dyt (*fcn)(dyt, dyt, dyt, dyt, dyt, dyt, dyt));
dyt dyFunc8(dyt (*fcn)(dyt, dyt, dyt, dyt, dyt, dyt, dyt, dyt));

dyt dyMethod0(dyt (*fcn)(dyt self));
dyt dyMethod1(dyt (*fcn)(dyt self, dyt a));
dyt dyMethod2(dyt (*fcn)(dyt self, dyt a, dyt b));
.... (up to 8)
dyt dyMethodN(dyt (*fcn)(dyt self, dyt rest));
dyt dyMethod1N(dyt (*fcn)(dyt self, dyt a, dyt rest));
dyt dyMethod2N(dyt (*fcn)(dyt self, dyt a, dyt b, dyt rest));
.... (up to 8)


Objects:

dyt dyObject();
dyt dyObjectParent(dyt parent);
Create a new prototype object. In the former case, the object is empty, and
in the a latter, a parent is defined. The parent will be used in a 'parent'
delegate slot.

dyt dyGet(dyt obj, char *sym);
Get a slot from an object.
Returns UNDEFINED if the slot was not found.
May recurse into delegates in order to locate the slot.

void dySet(dyt obj, char *sym, dyt val);
Set a slot in an object.
Recursion may be used in order to locate the slot, and if found, it is set
in the object in which it is defined. If not found, it is bound within the
current object.

void dyBind(dyt obj, char *sym, dyt val);
This is similar to dySet(), but differs primarily in the way in which slots
are assigned. In the case of dyBind, if the slot does not exist within the
current object, it is created (rather than recursing and assigning).

void dyBindMethod(dyt obj, char *sym, dyt mth);
This will bind a method in the current object. This will behave differently
than the normal 'set' or 'bind' operations, in that multiple versions of a
method may be bound in an object. In this case, they will be merged into a
kind of 'multi-method', which will dispatch based on args count. As such,
Set() and Bind(), will not perform such merging (but will simply replace the
method, including multi-methods).

void dyDefMethod(dyt obj, char *sym,
dyt (*fcn)(dyt self, dyt *args, int nargs));
dyt dyDefMethod0(dyt obj, char *sym,
dyt (*fcn)(dyt self));
dyt dyDefMethod1(dyt obj, char *sym,
dyt (*fcn)(dyt self, dyt a));
dyt dyDefMethod2(dyt obj, char *sym,
dyt (*fcn)(dyt self, dyt a, dyt b));
.... (up to 8)

void dyDefMethodN(dyt obj, char *sym,
dyt (*fcn)(dyt self, dyt rest));
dyt dyDefMethod1N(dyt obj, char *sym,
dyt (*fcn)(dyt self, dyt a, dyt rest));
dyt dyDefMethod2N(dyt obj, char *sym,
dyt (*fcn)(dyt self, dyt a, dyt b, dyt rest));
.... (up to 8)

Define a method for an object.
This will be similar to the above, but will accept a raw function pointer
instead of a method object.

The N-forms will fold the rest of the args into a list.


dyt dyGetDelegate(dyt obj, char *sym);
void dySetDelegate(dyt obj, char *sym, dyt val);
Will get or set the value of an object's delegate.

dyt dyGetParent(dyt obj);
void dySetParent(dyt obj, dyt val);
Special cases of the above, but are specialized to the 'parent' delegate.

dyt dyCall(dyt obj, char *sym, dyt *args, int nargs);
dyt dyCall0(dyt obj, char *sym);
dyt dyCall1(dyt obj, char *sym, dyt a);
dyt dyCall2(dyt obj, char *sym, dyt a, dyt b);
.... (up to 8)
dyt dyCallN(dyt obj, char *sym, dyt args);
dyt dyCall1N(dyt obj, char *sym, dyt a, dyt args);
dyt dyCall2N(dyt obj, char *sym, dyt a, dyt b, dyt args);
.... (up to 8)


Call a method within an object, and return the result.
Returns UNDEFINED if the method was not found.
Otherwise, this will return the return value of the method.


A simple example:

dyt testMath_doesNotUnderstand(dyt self, dyt sym, dyt rest)
{
if(!strcmp(sym, "pyth"))
return(dysqrt(dyadd(
dysqr(dycar(rest)),
dysqr(dycadr(rest)))));
return(NULL);
}

....
dyt obj, v;

obj=dyObjectParent(dyTop());
dySetDelegate(dyTop(), "testMath", obj);
dyDefMethod1N(obj, "doesNotUnderstand", testMath_doesNotUnderstand);

v=dyCall2(dyTop(), "pyth", dyint(3), dyint(4));