29. Bigloo A practical Scheme compiler User manual for version 4.0a January 2013 -- Bigloo Libraries

main page

top: | Bigloo A practical Scheme compiler User manual for version 4.0a January 2013

Bigloo Libraries

Chapters

Bigloo libraries are collections of global bindings (global variables and global functions). Bigloo libraries are built on the top of the host operating system (e.g. Unix) libraries. Because Bigloo uses modules, a library is not only a bundle of compiled codes and memory locations. A Bigloo library is split into several files:

- one heap that describes the variables and functions of the library.
- several host library files (safe and unsafe versions of the compilation and also eval libraries that contain the code that binds the variables and functions to the evaluator).
- possibly, C header files.
- possibly, an initialization file.

Let's consider, for example, a library that implements the `format` Common Lisp facility. Let's suppose we name this library `bformat` and that the library number is `1.0`. Using a Unix machine, the Bigloo library will consist of the following files:

- `bformat.heap`: the heap file.
- `bformat.init`: the initialization file.
- `libbformat_s-1.0.a`, `libbformat_s-1.0.so`, `libbformat_u-1.0.a`, `libbformat_u-1.0.so`, `libbformat_eu-1.0.so`, and `libbformat_es-1.0.so`:: the Unix library files. The file names with a `_u` are libraries compiled in unsafe and optimized mode. By convention the library using the `_s` suffix are safe libraries, `_p` are profiling libraries, `_d` debug libraries, `_es` and `_eu` eval libraries.
- `bformat.h`: an include file.

### 29.1 Compiling and linking with a library

From the user standpoint, using a library can be made two ways:

- Using the Bigloo `-library `lib-name`` option where `lib-name` is the name of the Bigloo library (not the name of one of the Unix files implementing the library). The name of the library must be lower case. For instance:

      $ bigloo foo.scm -library bformat

- Using the module clause `library`. This second solution avoids using a special compilation option. For instance, this module will automatically compile and link with the `bformat` library:

      (module foo
         (library bformat))

      ...
      (format ...)

When a Bigloo library ``lib`` is used, Bigloo automatically searches for a file called ``lib`.init` (the "init file"). If such a file exits, it is loaded at compile-time. For instance, the init file may be used to specify compilation flags or to define macros used by the compiler. The initialization file may affect any of the global parameters of the Bigloo compiler. For instance, a Bigloo library supporting SSL connections would likely need a native library. Setting the compiler variable `*ld-post-options*` has this effect. For instance, one may define an initialization file such as:

    (cond-expand
       (bigloo-compile
        (set! *ld-post-options* (string-append "-lssl " *ld-post-options*)))
       (bigloo-eval
        #unspecified))

When a Bigloo library `lib` is used, the Bigloo linker automatically looks at a library to be linked against the application. The name of the file containing the library depends on the operating system and the back-end used. For instance, under Unix, for a library called NAME, the Bigloo linker searches for a file called `libNAME_[s|u]-VERSION.a` or `libNAME_[s|u]-VERSION.DYNLIB-SUFFIX` in the compilation linker path when using the native back-end. It searches for a file `NAME_[s|u]-VERSION.zip` when the JVM back-end is used.
This default NAME can be overridden in the initialization file. The function `declare-library!` associates a Bigloo library name and a system name.

declare-library! ident [attributes] | library procedure

All the attributes are optional.

- `version:` the version number of the library. This defaults to the Bigloo version number.
- `basename:` the base of the filename containing the library. This defaults to the library name.
- `srfi:` a list of symbols denoting the SRFI 0 features implemented by this library. Registered SRFIs may be tested by the `cond-expand` form (see SRFIs). This defaults to an empty list.
- `dlopen-init:` a function to be invoked when the library is dynamically loaded using the function `dynamic-load`. This defaults to `#f`.
- `module-init:` a module to be initialized when the library is loaded. This defaults to `#f`.
- `eval-init:` a module to be initialized for binding the library exports in the interpreter. This defaults to `#f`.
- `class-init:` the JVM or .NET class name containing the module to be initialized. This defaults to `#f`.
- `eval-init:` the JVM or .NET class name containing the module to be initialized for eval. This defaults to `#f`.
- `init:` a function to be invoked when a library is loaded. This defaults to `#f`.
- `eval:` a function to be invoked when a library is loaded by the interpreter. This defaults to `#f`.
- `eval:` a function to be invoked when a library is loaded by the interpreter. This defaults to `#f`.

Examples:

- The following declares a library named `foo`. When loaded, the Bigloo runtime system will seek file named `libfoo_s-3.4a.so`, `libfoo_u-3.4a.so`, `libfoo_es-3.4a.so`, and `libfoo_eu-3.4a.so`.

      (declare-library! 'foo)

- The following declares a library named `pthread`. When loaded, the Bigloo runtime system will seek a file named `libbigloopth_s-1.1a.so`, `libbigloopth_u-1.1a.so`, `libbigloopth_es-1.1a.so`, `libbigloopth_eu-1.1a.so`. Once the library loaded, the SRFI-0 features `pthread` and `srfi-18` will be bound. When loading the library, the two modules `__pth_thread` and `__pth_makelib` will be initialized. In the JVM version these modules are compiled in the classes `"bigloo.pthread.pthread"` and `"bigloo.pthread.make_lib"`.

      (declare-library! 'pthread
                        :basename "bigloopth"
                        :version "1.1a"
                        :srfi '(pthread srfi-18)
                        :module-init '__pth_thread
                        :module-eval '__pth_makelib
                        :class-init "bigloo.pthread.pthread"
      		  :class-eval "bigloo.pthread.make_lib")

library-translation-table-add! ident name | library procedure

library-translation-table-add! ident name version | library procedure

library-translation-table-add! ident name version :dlopen-init initsym | library procedure

The function `library-translation-table-add!` is obsolete. It should no longer be used in new code. It is totally subsumed by `declare-library!`. The function `library-translation-table-add!` is still documented for enabling readers to understand old Bigloo source code.
This function registers a `name` for the library `id`. An optional `version` can be specified. The optional named argument `dlopen-init` gives the base name of the initialization entry point of a library.
Imagine that we would like to name our `bformat` library `bigloobformat`. This can be achieved by adding the following expression in the initialization file.

    (library-translation-table-add! 'bformat "bigloobformat")

Using this translation, on a Unix platform, the library used during the linking will be named: `libbigloobformat_s-<BIGLOO-VERSION>.a`. In order to change the `<BIGLOO-VERSION>` to another suffix, such as `1.0`, one may use:

    (library-translation-table-add! 'bformat "bigloobformat" "1.0")

In such a case, the library searched will be named `libbigloobformat_s-1.0.a`.
Specifying a `#f` prevents the insertion of any suffix. Hence,

    (library-translation-table-add! 'bformat "bigloobformat" #f)

instructs the compiler to look at a library named `libbigloobformat_s.a`.

### 29.2 Library and inline functions

It is illegal for libraries to include inline functions that make use of new foreign types. By "new foreign type", we mean foreign types that are defined inside the library. A library may contain inline functions but these inline functions must not call functions using foreign types in their prototypes. Including inline functions making use of foreign C types will make the compiler fail when compiling user code, prompting type errors. A library may contains non-inline functions that make use of new foreign types.

### 29.3 library and eval

The function `library-load` loads a library in the interpreter.

library-exists? ident . path | library procedure

Checks if the library `ident` exists for the current back-end.
The regular Bigloo library paths are scanned unless optional `path`s are sent to the function.

bigloo-library-path | library procedure

bigloo-library-path-set! | library procedure

These functions get and set the default path (a list of strings) for loading libraries.

library-load ident . path | library procedure

Loads a library in the interpreter. In addition to dynamically loading the library, this function tries to load the `_es` version of the library if it is linked against the safe Bigloo library version or the `_eu` version if it is linked against the unsafe version of the Bigloo library.
Searches for libraries occur in the regular Bigloo library paths unless optional `path`s are sent to the function.
This version may be used for automatically exporting bindings to the interpreter. In general, the `_es` and `_eu` libraries are simple libraries that contain only one module, the module that is used to build the heap-file. For instance, let's consider an implementation of a library for SSL programming. This library is composed of a single implementation module `__ssl_ssl`. The library is build using a heap file:

    (module __ssl_makelib
       (import __ssl_ssl))

Changing this file for:

    (module __ssl_makelib
       (import __ssl_ssl)
       (eval   (export-all)))

enables the construction of the `_es` and `_eu` libraries.
When the system loads a dynamic library, it initializes it. For that it expects to find initialization entry points in the dynamic libraries that are named after the library's name. More precisely, for the `LIB_s` library, the loader seeks the entry point named `"LIB_s"` and for the `LIB_es`, it seeks `"LIB_es"`. The name of the initialization entry of a library can be changed using the `declare-library!` function. If that named is changed, one module of the library must contain an `option` module clause that sets the variable `*dlopen-init*` with the name of the initialization entry point.
Since Bigloo 3.1a, the runtime system supports a better way for initializing libraries. Initialization modules can be associated with a library. When loaded, these modules are automatically initialized. This new method fits harmoniously with the Bigloo initialization process and it relieves users from any requirement to annotate the source code of the library. For instance, if a library initialization file contains the following declaration:

    (declare-library! 'foo :module-init 'foo)

Then, the library must implement the `foo` module.

    (module foo
      (import ...)
      ...)

In addition if the library binds variables, functions, or classes in the interpreter then, an `eval-init` clause must be added to the class declaration:

    (declare-library! 'foo :module-init 'foo :eval-init 'foo-eval)

Then, the module `foo-eval` must be implemented in the `libfoo_es` and `libfoo_eu` libraries.

    (module foo-eval
      (import ...)
      (eval (export-all)))

The standard distribution contains examples of such constructions. In particular, the multi-threading libraries `pthread` and `fthread` use this facility.

### 29.4 library and repl

It is possible to implement a "read-eval-print-loop" that is extended with the facilities implemented inside a library. In order to make the variables, functions, and classes of a library visible from the interpreter, the eval `library` module clause has to be used. (see Module Declaration) For instance, here is a module that implements a "repl" with the `format` facility available:

    (module format-repl
       (eval (library bformat))
       (library bformat))

    ;; a dummy reference to a facility of the format library
    (let ((dummy format))
       (repl))

Alternatively, libraries can be explicitly loaded using the `library-load` function such as:

    (module format-repl)

    ;; a dummy reference to a facility of the format library
    (let ((dummy format))
       (eval '(library-load bformat))
       (repl))

### 29.5 Building a library

Build Bigloo libraries require several steps that are explained in this section. This section shows how to create static and dynamic (or shared) libraries. However not that creating a dynamic library highly dependent on the host operating system. Users willing to create dynamic libraries on other operating systems should use the `api` directory of the Bigloo source code tree as an example.

- The first step is to build a library heap. This is achieved using a special compilation mode: `-mkaddheap -mkaddlib -addheap -heap-library <ident>`. That is, for your library you have to create a heap associated source file that imports all the binding you want in your library. The heap source file must be excluded from the source files that will be used to build the host library.
  Suppose we have a unique source file, `bformat.scm` for our library. The module clause of this source file is:

      (module __bformat
         (export (bformat fmt::bstring . args)
                 bformat:version))

      (define (bformat fmt . args)
         (apply format (string-replace fmt #\% #\~) args))

      (define bformat:version 1.0)

  Prior to compiling the library, we have to create the heap associated file (let's name it `make_lib.scm`). This file could be:

      (module __make_lib
         (import (__bformat "bformat.scm"))
         (eval (export-all)))

  Building it is simple:

      bigloo -unsafe -safee -q -mkaddheap -mkaddlib -heap-library bformat \
           make_lib.scm -addheap bformat.heap

  The options `-mkaddheap` and `-mkaddlib` tell Bigloo that it is compiling an heap associated file. The option `-addheap` tells Bigloo the name of the heap file to be produced. The option `-heap-library` instructs the compiler for the library name to be included inside the heap file. This name is used for checking versions at run-time.
- The second step is to compile all the library source file. These compilation must be done using the `-mkaddlib` compilation mode. For example:

      bigloo -O3 -unsafe -safee -mkaddlib       \
         -cc gcc -fsharing -q -rm               \
         -unsafev bformat.scm -o bformat_u.o -c
      bigloo -O3 -mkaddlib -g -cg -cc gcc       \
         -fsharing -q -rm                       \
         -unsafev bformat.scm -o bformat.o -c

  The first compilation produces the unsafe version the second the produced the debugging version.
- The third step is to build the host operating system libraries. There is no portable way to do this. This operation may looks like:

      ar qcv libbigloobformat_s-1.0.a bformat.o
      ranlib libbigloobformat_s-1.0.a
      ld -G -o libbigloobformat_s-1.0.so bformat.o -lm -lc
      ar qcv libbigloobformat_u-1.0.a bformat_u.o
      ranlib libbigloobformat_u-1.0.a
      ld -G -o libbigloobformat_u-1.0.so bformat_u.o -lm -lc

- The fourth step consist in creating the `bformat_es` and `bformat_eu` libraries for eval. For the unsafe version we use:

      bigloo -O3 -unsafe -safee -mkaddlib       \
         -cc gcc -fsharing -q -rm               \
         -unsafev make_lib.scm -o make_lib.o -c
      ld -G -o libbigloobformat_eu-1.0.so make_lib.o -lm -lc
      ar qcv libbigloobformat_eu-1.0.a make_lib.o
      ranlib libbigloobformat_eu-1.0.a

  For the safe version we do:

      bigloo -O3 -mkaddlib              \
         -cc gcc -fsharing -q -rm               \
         -unsafev make_lib.scm -o make_lib.o -c
      ld -G -o libbigloobformat_es-1.0.so make_lib.o -lm -lc
      ar qcv libbigloobformat_es-1.0.a make_lib.o
      ranlib libbigloobformat_es-1.0.a

- The last step is to create an initialization file `bformat.init`:

      (declare-library! 'bformat
         :version "1.0"
         :srfi '(bformat)
         :basename "bigloobformat"
         :module-init '__bformat
         :module-eval '__make_lib
         :class-init "bigloo.bformat.__bformat"
         :class-eval "bigloo.bformat.__make_lib")

At this time, you are ready to use your library. For that, let's assume the file `foo.scm`:

    (module foo
       (library bformat))

    (bigloo-library-path-set! (cons (pwd) (bigloo-library-path)))
    (print (bformat "Library path: %a" (bigloo-library-path)))

    (eval '(library-load 'bformat))
    (repl)

It can be compiled and executed with:

    bigloo foo.scm -L . -copt -L.
    LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH ./a.out

The Bigloo distribution contains library exemplars that should probably considered as a departure point for new libraries.

### 29.6 Library and modules

A Bigloo library may be composed of several Bigloo modules (even if in our example only one module was used). The modules composing the library are free to import each other. Nevertheless, someone designing a Bigloo library should be aware that Bigloo importation creates dependences between modules. A module `mod1` that imports a module `mod2` depends on `mod2` because `mod1` requires `mod2` to be initialized (i.e. `mod1` calls to the initialization function of `mod2`). The result is that using `import` clauses inside modules composing a library may create a lot of dependencies between the object files that are used to build the associated Unix library. Dependencies should be avoided because they make the Unix linkers unable to produce small stand-alone programs. Instead of `import` clauses, `use` clauses should be preferred. `Use` clauses do not create dependencies because a module `mod1` that `use`s a second module `mod2` does not require `mod2` to be initialized. Of course, it may happen situations where the initialization is mandatory and thus, the `import` must not be replaced with a `use` clause. The source code of the Bigloo library makes use of `import` and `use` clauses. The Bigloo standard library should be studied as an example.

### 29.7 Library and macros

Bigloo libraries can export macros, expanders, and syntaxes but these must be handled carefully. Macros (these also applies to expanders and syntaxes) exported by modules are not visible by client code. Exported macros have to be placed inside the initialization file. For instance, if we change the definition of `bformat.init` file for:

    (declare-library! 'bformat
       :version "1.0"
       :srfi '(bformat)
       :basename "bigloobformat"
       :module-init '__bformat
       :module-eval '__make_lib
       :class-init "bigloo.bformat.__bformat"
       :class-eval "bigloo.bformat.__make_lib")

    (define-expander BFORMAT
       (lambda (x e)
          (match-case x
             ((?- (? (lambda (s) (and (string? s) (not (string-index s #\%))))) . ?a
    )
              `(string-append ,@(cdr x)))
             (else
              `(bformat ,@(map (lambda (x) (e x e)) (cdr x)))))

At compile time the macro BFORMAT will be declared. Hence, we can change the definition of `foo.scm` for:

    (module foo
       (library bformat))

    (bigloo-library-path-set! (cons (pwd) (bigloo-library-path)))
    (print (BFORMAT "library path: %a" (bigloo-library-path)))

    (eval '(library-load 'bformat))
    (repl)

### 29.8 A complete library example

For the means of an example let's suppose we want to design a Bigloo library for 2d points. That library is made of three implementation files: two C files, `cpoint.h` and `cpoint.c` and one Scheme file `spoint.scm`. Here are defined the three files:
`cpoint.h`:

    struct point_2d {
       double x, y;
    };

`cpoint.c`:

    #include <stdio.h>
    #include "cpoint.h"

    int print_point_2d( struct point_2d *pt ) {
       printf( "<point-2d: %g, %g>", pt->x, pt->y );
    }

`spoint.scm`:

    (module __point
       (include "spoint.sch")
       (extern  (include "cpoint.h"))
       (export  (make-point::s-point_2d* ::double ::double)
                (print-point ::s-point_2d*)
                (point? ::obj)))

    (define (make-point::s-point_2d* x::double y::double)
       (s-point_2d* x y))

    (define (print-point p::s-point_2d*)
       (print_point_2d p))

    (define (point? obj::obj)
       (s-point_2d*? obj)
       obj)

`makelib.scm`: We want our library to be composed of the whole exported Scheme functions. Thus the file to build the heap library could look like:

    (module __point_makelib
       (import __point)
       (eval (export-all)))

`point.init`:Let's suppose that the `point` library requires the `libposix` library. This means that any file linked with the `point` library needs to be also linked with the `posix` library. Furthermore, programs making use of the `point` library needs to include the `point.sch` file. That Scheme file needs in turn the C file `point.h` otherwise the produced C files won't compile. The need for the `libposix` library and for the `point.h` file may be specified inside the `point.init` file. For our current library, the `point.init` file could look like:

    (declare-library! 'point
                      :basename "point"
                      :srfi '(point)
                      :eval-init '__point_makelib)

    (set! *ld-options*
          (string-append "-L/usr/lib " *ld-options*))

    (set! *bigloo-user-lib*
          (cons "-lm" *bigloo-user-lib*))

    (set! *additional-include-foreign*
          (cons "cpoint.h" *additional-include-foreign*))

    (define-macro (point x y)
       `(make-point ,x ,y))

This file updates some compilation variables (`*ld-options*`, `*bigloo-user-lib*`, `*additional-include-foreign*`) and defines a macro: `point`. Because the `point.init` file will be loaded each time a compilation require the `point` library is spawned, user code are allowed to use the `point` macro. Here is an example file making use of the `point` library:
`example.scm`

    (module example)

    (let ((p (point 2.9 3.5)))
       (print "point?: " (point? p))
       (print "point?: " (point? 4))
       (print-point p)
       (print "done..."))

To conclude that example here is the `Makefile` used to compile the `point` library, heap file and one example.

    # bigloo flags
    BIGLOO          = bigloo
    RELEASE		= `$(BIGLOO) -eval '(begin (print *bigloo-version*) (exit 0))'`
    BHEAPFLAGS      = -unsafe -q -mkaddheap -mkaddlib -v2 -heap-library point
    BCOMMONFLAGGS   = -mkaddlib -fsharing -q $(VERBOSE)        \
                      -copt '$(CCOMMONFLAGS)' -cc $(CC)
    BSAFEFLAGS      = $(BCOMMONFLAGGS) -cg -O3 -g -cg -unsafev \
                      -eval '(set! *indent* 4)' -rm
    BUNSAFEFLAGS    = $(BCOMMONFLAGS) -O4 -unsafe

    # cigloo flags
    CIGLOO          = cigloo

    # cflags
    CC              = gcc
    CCOMMONFLAGS    = -I.
    CSAFEFLAGS      = $(CCOMMONFLAGS)
    CUNSAFEFLAGS    = $(CCOMMONFLAGS) -O2

    # library objects
    SAFE_OBJECT     = olib/spoint.o olib/cpoint.o
    UNSAFE_OBJECT   = olib_u/spoint.o olib_u/cpoint.o

    all: .afile heap lib example

    .afile: spoint.scm makelib.scm
    	bglafile $^ > $@

    heap: point.heap

    point.heap: spoint.sch spoint.scm
    	$(BIGLOO) $(BHEAPFLAGS) makelib.scm -addheap point.heap

    lib: lib_u lib.a

    lib.a: olib $(SAFE_OBJECT)
    	ar qcv libpoint_s-$(RELEASE).a $(SAFE_OBJECT)

    lib_u: olib_u $(UNSAFE_OBJECT)
    	ar qcv libpoint_u-$(RELEASE).a $(UNSAFE_OBJECT)

    olib:
    	mkdir olib

    olib_u:
    	mkdir olib_u

    olib_u/spoint.o olib/spoint.o: spoint.scm
    	$(BIGLOO) $(BSAFEFLAGS) $(<F) -o $*.o -c

    olib_u/cpoint.o olib/cpoint.o: cpoint.c
    	$(CC) $(CSAFEFLAGS) $(<F) -o $*.o -c

    spoint.sch: cpoint.h cpoint.c
    	cigloo $^ > $@

    example: heap lib
    	$(BIGLOO) -v2 -L . -library point \
                -static-bigloo example.scm -o example

    clean:
    	-/bin/rm -f point.heap
    	-/bin/rm -f spoint.sch spoint.c
    	-/bin/rm -fr olib olib_u
    	-/bin/rm -f example example.c example.o
    	-/bin/rm -f libpoint_s-$(RELEASE).a libpoint_u-$(RELEASE).a

This Html page has been produced by Skribe.
Last update Thu Jan 24 17:08:36 2013.