Overview

LPSG is a Common Lisp library for rendering graphics using OpenGL. It presents a programming interface that allows it to implement optimizations that are important on modern graphics hardware. LPSG's target applications include 3D content creation programs, games, and geospatial viewing programs -- in short, any domain in which excellent interactive rendering of complex scenes is required. Another important class of applications that LPSG will support is 2D applications such as text editors which need fast, portable rendering.

Note: LPSG is a work in progress. Throughout this manual future work or features will be indicated [like this.]

While LPSG does provide classes and functions for common 3D rendering tasks, its essential purpose is to aid the creation of fast programs that use OpenGL. A typical application that uses LPSG will specialize a few LPSG public classes and generic functions in order to inject some OpenGL code, such as a custom shader program, into LPSG's rendering code.

The main optimization attempted by LPSG is to reduce the memory traffic between the CPU and GPU. It does this by storing the user's data in OpenGL buffer objects which normally reside on the GPU. Most updates to a graphics view, such as matrices that specify eye or object positions, are small; these are handled through a system of parameters called usets. On the other hand, LPSG does [will] support efficient methods for uploading large updates of user data.

Another important optimization is reducing CPU cache misses during rendering. This is often caused by traversing large tree-like structures or scene graphs in the user's application. LPSG uses a different approach: the application submits graphics objects to LPSG, which is from then on responsible for traversing the objects and displaying them. LPSG can use data structures that are more efficient than the application's, [and much of the traversal may even be moved to the GPU.] Newer versions of OpenGL contain interfaces that can reduce the number of cache misses inside the driver itself, [LPSG will take of advantage of them.]

Another optimization which was historically crucial, and is still helpful, is reducing the number of state changes in the graphics hardware. Changing the rendering target, current shader program, or set of bound textures requires reinitializing a lot of hardware state and may cause a stall on the GPU. [LPSG will sort the rendering order of graphics objects to minimize state changes.]

A side effect of all these optimizations is to reduce the number of calls to the OpenGL API. This is good for OpenGL performance in general, as each OpenGL call requires work from the driver to at least validate it and check for errors. This is especially good in Common Lisp, as "foreign function calls" can be quite expensive.

Conventions

LPSG is described in terms of protocols and protocol classes. A protocol is a set of generic functions that collaborate to do something. The functions in a protocol must restrict some of their arguments to classes that have certain properties. These classes are protocol classes, abstract classes that serve only as superclasses. A protocol class defines accessors, which behave like normal slot accessors, although they can take more than one argument. It also defines normal generic functions that are considered to be part of the class, even though classes don't implement functions and methods in Common Lisp.

If a class inherits from a protocol class, then there must exist applicable methods for all the accessors and generic functions defined by the protocol class when the derived class appears as an argument to them. LPSG provides mixins that contain slots whose accessor methods implement the protocol classes' accessor generic functions.

Protocol classes are defined using the define-protocol-class macro:

LPSG provides instantiable classes that are subclasses of its protocol classes. These have names beginning with "standard-" or "simple-" and are documented along with the protocol classes.

High level LPSG

LPSG contains a renderer class that controls all rendering to the screen. The user creates shapes containing per-vertex attributes, such as position or color data. The shape also contains an effect object which controls its appearence on the screen.

Once a shape has been prepared, it is submitted to the renderer using the generic function submit. This invokes the submit-with-effect generic function which, dispatching on its renderer, shape, and effect arguments, creates the objects that are used in the actual OpenGL rendering and places them on various queues. These are used when the user calls the renderer's draw method; at that time OpenGL objects are created on the GPU, any necessary initialization is performed, and shapes are rendered. On future calls to draw, these objects will be rendered again. The user effectively gives up ownership of a shape object until the retract method is called on it. This removes the objects created in the submit call from the renderer's queues and destroys any now-uneeded OpenGL objects.

Scenes in 3D applications are very dynamic; if they weren't, there wouldn't be much point in displaying them in 3D. At the least, it is possible for the viewpoint to change at every redraw of the screen. The geometric objects in the scene may be animated as well, and these are often evaluated within systems of nested coordinate systems. To support this dynamism, LPSG implements a system of parameters that are connected in dependency chains. These incremental-node objects use lazy evaluation to supply values that affect OpenGL rendering. These values are most often used to set the uniform variables in OpenGL shader programs. LPSG aggregates such variables into sets, in order to choose the optimal way to upload them to OpenGL; these sets are called "uniform sets" or, throughout LPSG, "usets."

Getting Started with LPSG

1. Prerequisites

LPSG and its example programs require OpenGL drivers that support version 3.3 at a minimum. It might work in older versions of OpenGL with some hacking. It has been developed using SBCL on x86_64 Linux and Clozure Common Lisp on ARM Linux and Windows.

2. Dependencies and Quicklisp

LPSG depends on several Common Lisp packages, including cl-opengl for interfacing with OpenGL. The easiest way to install these dependencies is to use @link[uri="https://www.quicklisp.org/beta/"'(Qucklisp). LPSG itself is not [yet] in Quicklisp, so the easiest way to use Quicklisp is to put the LPSG source in your quicklisp/local-projects directory and type

(ql:quickload :lpsg)

to load LPSG and its Common Lisp dependencies.

3. Examples

LPSG comes with some simple example programs in the examples subdirectory:

• cube.lisp - draw cubes
• texture.lisp - draw a textured cube

The Renderer

The renderer class that controls all rendering to the screen. Generic functions that dispatch on renderer ultimately perform all OpenGL calls. open-renderer performs any OpenGL initialization needed; close-renderer cleans up when the application is finished rendering.

The user calls the submit method to register objects with a renderer. Geometric objects, called "shapes," that have been submitted will then be rendered when the draw generic function is called. Submitted objects persist between calls to draw.

The retract generic function removes an object from a renderer. It will not be rendered anymore, and any OpenGL objects used by it will be deallocated if not used by any other LPSG objects. This deallocation is not immediate because it relies on garbage collection to notice when an object is no longer used.

Attributes and shapes

The shape class represents geometry that is rendred by OpenGL. In that sense it contains a collection of vertex attributes i.e. the per-vertex data used in the rendering. This might include a position in space (coordinates); a surface normal vector, colors, texture coordinates, or anything else in this modern world of shaders that could possibly be stored per-vertex in a shape. Each vertex attribute is stored in a vertex-attribute object.

Buffers

The vertex attributes used to render a shape are stored in a vertex buffer objects(VBO) within OpenGL. This object often resides in the GPU's memory. Different vertex attributes from different shapes can be stored in the same buffer object. LPSG references them with a buffer-area object that describes the location and format of data within a buffer.

The data in a buffer object is in the format required by the GPU and not easily accessible from a program running on the CPU. It is convenient for a Lisp program to manipulate shape attributes in Lisp arrays. The mirrored-buffer-resource class, which is a subclass of buffer-area, describes the location and layout of data in a Lisp array that will be uploaded by LPSG into a buffer. The source data of a mirrored-buffer-resource is described in terms of the elements of array, whereas the data location and format in the target buffer is defined in terms of bytes.

The object stored in the buffer slot of a buffer-area object is of class gl-buffer, described in the OpenGL Objects and State section. It can be shared among many buffer-area objects, thus storing their data in a single buffer object in OpenGL.vertex-attribute is a subclass of mirrored-buffer-resource.

shape contains an object called a "drawable" which describes the specific shape i.e., whether it is a collection of points, lines, or triangles, how many of these objects there are, and also whether the shape is indexed or not. This refers to whether the vertex attributes are just straight arrays of one value for every vertex in the shape, or whether the individual parts of the shape are described by indices which are indexes into the arrays of vertex attributes. Indexed shapes are important because not only do they offer a big size savings on meshes where a lot of the vertices would be repeated in the shape, they also enable another class of optimizations where the vertex attributes can be shared among different shapes.

An object called an effect controls the appearance of a shape. Its role is described in the "Submit Protocol" section.

A shape's slots may not be changed after it has been submitted to the renderer. The individual parts of the shape are not necessarily immutable; the drawable part is mutable, and one can change the number of primitives drawn, which might be useful to selectively draw parts of a shape. The vertex attributes can be mutable. It is important to be able to upload new data for many kinds of dynamic animation effects. The environment object is immutable, but the rendering it controls is influenced by uset parameters that attached to the shape via the incremental computation system, decribed next.

Incremental computation

One of the goals of LPSG is to optimize the amount of memory traffic to the GPU. To that end, LPSG provides a system of lazy evaluation, called "incremental nodes," which ultimately connects to the classes representing OpenGL structures and feeds them values. An application may change the value of a parameter; through a chain of intermediate connected incremental nodes, which have inputs (called "sources") and an output (called a "sink"), this causes a value to be marked as invalid. When that value is actually needed, the chain of incremental nodes that calculate the value is traversed and updated. Incremental nodes that calculate a value based on their inputs, called computation-nodes, store the result, so the value is not recomputed if it is not invalid.

LPSG calls the generic function notify-invalid-input to mark a node as invalid. This function can be specialized to perform other actions at that time e.g., putting a node on a queue to perform an upload to the GPU at a later time.

An example:

(progn
  (defclass plus-node (computation-node computation-node-mixin source-sink-mixin)
    ())

  (defmethod compute ((node plus-node))
    (let ((arg1 (input-value node 'arg1))
          (arg2 (input-value node 'arg2)))
      (+ arg1 arg2)))

  (defclass mult-node (computation-node computation-node-mixin source-sink-mixin)
    ())

  (defmethod compute ((node mult-node))
    (let ((arg1 (input-value node 'arg1))
          (arg2 (input-value node 'arg2)))
      (* arg1 arg2)))

  (defparameter *source1* (make-instance 'input-value-node))
  (defparameter *source2* (make-instance 'input-value-node))

  (defparameter *plus-node* (make-instance 'plus-node))
  (defparameter *mult-node* (make-instance 'mult-node))

  (setf (input *plus-node* 'arg1) *source1*)
  (setf (input *plus-node* 'arg2) *source2*)

  (setf (input *mult-node* 'arg1) *source1*)
  (setf (input *mult-node* 'arg2) *source2*)

  (setf (value *source1*) 4)
  (setf (value *source2*) 8)

  (defparameter *mult-node2* (make-instance 'mult-node))
  (setf (input *mult-node2* 'arg1) *plus-node*)
  (setf (input *mult-node2* 'arg2) *mult-node*)
  ;; Value of *mult-node2* should be 384.
  (format t "~%*mult-node2*: ~S" (value *mult-node2*))

  (defparameter *mult-node3* (make-instance 'mult-node))
  (setf (input *mult-node3* 'arg1) *mult-node2*)
  (setf (input *mult-node3* 'arg2) 10)
  ;; Value of *mult-node3* should be 3840.
  (format t "~%*mult-node3*: ~S" (value *mult-node3*))
  )

source-node objects can produce any type of value.

Uniform sets

This is all not very useful in terms of writing 3D applications. because if you put a shape on the screen with all these immutable attributes, then you can't navigate in the scene! You can't move the objects, no animation, can't do anything. This problem is resolved in lpsg by specifying parameters that end up affecting the environment. The parameters are grouped into sets, which we call "uniform sets," or "usets" as is used throughout lpsg. The idea is that parameters that are related to each other are grouped into sets and can eventually be updated in OpenGL in the same time, or at with the same frequency, and can be stored together. Uniform sets comes from the name that variables have in shader programs: "uniform variables." These are variables that change slowly; they do not change in the course of the rendering of a single shape. Their values might change between different shapes. And this is how an environment object could be used to render different shapes on the screen because one parameter of the environment would have to be the model matrix, which specifies where the object is in space. This is a parameter that is different for every shape, and it could change during the execution of the user's program. Another example of a uset are the parameters describing the viewpoint and camera. These are usually know as the view matrix and projection matrix. -- As well as other matrices associated with the camera, such as an inverse, transposed matrix used to transform normal vectors.

Because incremental computation source-node objects can produce any type of value, they can also produce uniform sets. The LPSG user assigns standard-shape nodes as sinks for incremental computation nodes that produce uset values, using the name of the uset as the input name. A shape doesn't actually have a role at render time, but it serves as a placeholder for objects that will need to receive the uset values and update rendering parameters. The appropriate connections are made when the shape is submitted. [This may change when the incremental computation protocol is based more on the MOP and is a bit less dynamic in terms of input slots. Also, it makes more sense to attach incremental nodes to the effect object than to the shape.] The usets produced by incremental nodes should not be freshly allocated at each update, but should be updated in place. Environments currently rely on this, which is kind of brittle and should be fixed.

Textures

Textures are raster data which are most often used to map images onto polygonal faces in the rendered scene. They are treated differently from buffer objects by OpenGL because they are used differently. the individual texels of a texture are usually sampled in a 2D region rather than linearly, so a conventional cacheing strategy doesn't work well. Also, different resolutions of the texture are often stored in a mipmap to support filtering. Following the lead of OpenGL, LPSG provides classes that are not a buffer-area or mirrored-buffer-resource for managing textures.

There is an asymmetry in the LPSG interfaces, and their concrete implementations, for buffer-area and texture-area classes. The buffer area provides offset and stride parameters for accessing source data and packing them in a buffer for OpenGL. In contrast, texture-area and its subclasses provide x-offset, y-offset, and row-alignment parameters for accessing an arbitrary "rectangle" of data in a larger image. This reflects the functionality offered by OpenGL, as well as the different uses of buffer objects and textures.[A less excusable asymmetry currently exists in the implementation: the mirrored-buffer-resource only supports loading buffers from Lisp arrays, whereas raw-mirrored-texture-resource, the only subclass of texture-resource, only supports loading from "foreign" memory. This is due to pragmatism -- image loading libraries are likely to return image data as foreign memory -- as well as laziness. It will be fixed soon.]

The Submit Protocol

The effect object stored in a shape is not used directly to perform OpenGL rendering. It does, however, control how the rendering will be done. This is accomplished in the submit generic function, when a shape is submitted to the renderer. submit immediately calls another generic function, submit-with-effect, which can be specialized on its shape and effect arguments.

The protocol class effect is the superclass of all effect objects.

The attribute-map is an essential link between a shape and its on-screen representation. It maps the names of attributes in the shape to vertex attributes in an OpenGL shaper program.submit-with-effect creates objects that are used to make calls to OpenGL and to do rendering. These objects are similar to the attribute, shape and effect objects that the user manipulates, but they are less flexible and more specific to their role in OpenGL rendering. The include:

• attribute-set - an optimized representation of a shape's attributes;
• environment - An object created from the effect, containing OpenGL resources and parameters. see the "Environment" section;
• render-bundle - The rough equivalent of a shape, which contains an attribute-set and an effect.

submit-with-effect can create several render-bundle objects from a shape, each with a different environment(and attribute set, if necessary). These bundles are are added to render-queue objects, so they will be drawn at a later time. In this way submit-with-effect can arrange for a shape's geometry to be rendered several times with different environments.submit-with-effect also performs other important functions. Any object that refers to OpenGL objects may need to be finalized and should be placed on the "finalize" queue. "Finalization" here does not refer to garbage collector finalization, but to the actions required to create and intialize objects via calls to OpenGL. The generic function gl-finalize, described in the section "OpenGL Objects and State," is called on every object in the finalize queue.

All data that will be used by OpenGL must be scheduled for upload to the driver and GPU.

Uploading Data

The "upload queue" is an opaque object not directly accessible to the user. The upload of some types of objects can be optimized by grouping all uploads for a certain object e.g., a buffer object, together. The generic function schedule-upload is used to schedule the upload; this will happen after the finalize queue has been traversed, but before any shapes are drawn.

All these queues are traversed by the draw generic function, which is described in the "Draw protocol" section.

[The interfaces to queues are very ad hoc at the moment. Some queues are exposed as lists; others have a more abstract interface. This all needs to be abstracted so that queue contents can be maintained in sorted order.]

The finalize and upload queues are cleared after each call to draw, while the render queues are not. [Should some uploads be preserved i.e., have the option to be done every frame?]

Methods defined on the simple-effect class perform the basic actions of creating render bundles from a shape and scheduling any necessary finalization and upload.

Environments

The environment object contained in a render-bundle ultimately controls the appearance of the bundle.

The gl-state slot contains a graphics-state object that represents the OpenGL state in effect when the environment is used to render a bundle. This contains things such as the shader program, and the the bound textures, and is described in the OpenGL Objects and State section.

The usets slot contains the uniform set parameters for the shader program contained in the graphics state. These are kept up-to-date by the incremental computation mechanism, as the environment is a sink object that receives incremental results. There are different possible strategies for making uniform values available to shader programs. At present, LPSG uses the classic OpenGL glUniform family of functions to upload values into shader programs. [In the future, we want to support using Uniform Buffer Objects.]

OpenGL Objects and State

LPSG uses the cl-opengl library to call functions in the OpenGL "C" ABI. Most programs that use LPSG will not need to call OpenGL directly, but cl-opengl is recommended if needed. It provides good perfomance while mapping the OpenGL interface to Lisp in a natural way. For example, the OpenGL function glVertexAttribPointer is called as gl:vertex-attrib-pointer in Lisp. In this documentation, we refer to OpenGL functions by their Lisp name.

Any OpenGL rendering requires the creation of resources that are managed by the OpenGL driver. This typically follows a pattern of generating a name (a 32 bit integer) for the resource and then initializing it. When the resource is no longer needed, it is destroyed and the name can be reused by the driver. LPSG manages this process with a class called gl-object. Subclasses are defined for specific OpenGL resources such as textures and shader programs.

The OpenGL resource associated with a gl-object is not allocated or initialized when the gl-object is created via make-instance. [This is common graphics programming practice, and also is compatible with a future implementation of LPSG where OpenGL calls are made in a seperate thread from the user program.] The generic function gl-finalize is called by LPSG at an appropriate time to allocate the OpenGL resource and initialize it. This time is most often at the beginning of a call to draw, when objects in the renderer's finalize queue are visited, but gl-finalize may be called at other times too.

gl-finalize and gl-finalized-p are not restricted to take only subclasses of gl-object. Methods for those functions can be specialized on higher-level classes that containgl-object objects; such a method would make recursive calls to gl-finalize to initialize all its objects. An around method on gl-finalize calls gl-finalized-p, so it is not necessary to explicitly call gl-finalized-p before calling gl-finalize.

In Common Lisp implementations that support garbage collector finalization, LPSG will delete the OpenGL resource associated with a gl-object when that object is collected. For a variety of reasons it is not possible to perform the actual deletion in the context of a garbage collector finalization routine. Instead, LPSG uses weak pointers and auxilliary classes called proxy objects to defer the deletion to an appropriate time. [The user should be able to delete OpenGL objects explicitly; how / when does this happen?]

The graphics-state object stored in an environment contains OpenGL objects that should be bound in the OpenGL context, [as well as settings that are enabled,] when rendering is done using that environment.

The Draw Protocol

The generic function draw causes all submitted shapes to be rendered.

draw performs these actions:

• Process the finalize queue, calling gl-finalize on all objects contained in it;
• Upload data that has been registered with schedule-upload;
• Process the render stages, which results in OpenGL rendering.

Any rendering is done in OpenGL's default framebuffer.

Utilities