I’m building my own scientific software stack in C

I’ve written about writing C before, and building things you understand, so this project is a natural extension of both of those ideas.

Essentially, I write a lot of scientific software code - FEA, magnetics, fatigue, and fracture mechanics. Typically, I’ll use Python+NumPy+Matplotlib for most of those things, which often gets the job done just fine. When I need computational speed, I’ll write the backend in C or Rust, give it bindings, etc. I used to write a lot of Julia code and was pretty jazzed up about the language, but something about it just doesn’t sit all that well with me. I never got the hang of developing packages properly and having to reload the REPL when editing a module or redefining a struct (yes I know about Revise.jl) got tiring.

Anyways, I wanted to re-invent another wheel I guess, so I got the idea to build a top-to-bottom stack for my work in C. Python will still be my go-to for bindings (really hard to beat the combination of NumPy and Matplotlib), but I’ll do the rest in C. This project is a bit ambitious, but I’m clear-eyed about it probably taking me several years of very much part-time effort.

The core libraries I’m envisioning look like this:

• shepherd: core “standard” types and algorithms
• skipper: GUI and plotting
• flock: linear algebra and numerics

And a few projects to be built on top of them:

• sage: basic 3D FEA solver
• flawless: fatigue and fracture mechanics toolkit
• caspo: rewrite of Petoi’s OpenCat into something I can understand
• sonya: type 1 diabetes simulator

That’s a lot of stuff! But here’s the deal: I’m scoping it to:

• Only features that I plan to use
• Focus on correct and simple solutions - not world-class
• Incrementally building it over time when I have the enthusiasm and energy to do so

For me

Overall, the end goal with most of these things isn’t the tool itself, it’s fostering understanding. Specifically, my understanding. I’m not building these things for others to use, though they are open-source and can be used by anyone if they so choose. These will be simple, well-scoped libraries and tools that can be easily groked by anyone.

Foundations

• I’ll use the clang compiler, but avoid compiler intrinsics
• I’ll use macros sparingly
• I’ll use makefiles for build
• Documentation will be bundled with each codebase
• In general, functionality will be added only as it is needed

shepherd

I started this ‘standard’ or ‘core’ library when I first started moving away from Rust. I liked Rust’s Error, Option, and Result types, plus Vec. These things don’t quite work the same in C, but some good approximations do exist.

This isn’t meant to be a replacement for libc (it wraps fixed-width integers, system memory allocators like malloc, etc), but given its design I expect to be able to only need to link against it directly rather than #include libc headers throughout my other code.

Error

I defined an Error enum I can use across programs. I plan to include as many basic types of errors that make sense to bundle together. State 0 is OK, which mimics Rust, but offers a nice check. The thing this helps avoid is ambiguity when trying to parse what -2 or -10 means when a function returns a nonzerovalue on error.

Recipes

A lot of these types/functions are designed to be copy/pasted and edited for reuse with new datatypes. I call this a ‘recipe’ (perhaps that’s a common term for it?), which contrasts with Rust, OOP trying to make functions or data types modular and generic. If I need a new array with f16 values, I’ll make that array and supporting functions when I need it. No need to enable that ability ahead of time (hopefully I don’t regret saying that).

Maybe

I defined a Maybe recipe for defining a value that may or may not exist. This is basically the same idea as Rust’s Option, though it’s a bit more awkward to work with in C.

Arena

I’m learning about memory arenas and alternate methods of dynamic memory management. There’s a basic arena here, but I’m not sure how to use it well…

Perhaps a wrapper over malloc/calloc/free, if that ends up being useful (maybe that would end up being more work with no benefit, I’ll have to see).

Types

I created basic numeric types that mimic Rust and other languages: u32, f64, usize, etc. This reads a bit cleaner, I guess.

Strings

A string library to replace <string.h>. Conversion from and to shepherd strings and null-terminated libc strings.

Other ideas

• CSV read/write - I use this all the time
• TOML read/write - also very useful but I use less often
• Sorting algorithm - just to replace libc I guess; I figured that I should learn how to write one

flock

This is like a combo BLAS+LAPACK + sparse linalg all in one. My goal is to not compete with these libraries: I can’t. Instead, I want to learn how sparse data structures and linear solvers work.

Dense Linear Algebra

• Vector dot products and sums
• Matrix-vector multiply
• Matrix-matrix multiple (maybe just a naive 3 loop, or maybe I’ll introduce blocking?)
• LU solver
• Conjugate gradient solver with incomplete Cholesky preconditioner

Sparse Linear Algebra

• Sparse vectors and CSR matrices
• Matrix-vector multiply
• LDL solver (based on Tim Davis’ version)
• Conjugate gradient solver with incomplete Cholesky preconditioner

skipper

This is the one I’m most excited about, but having the most difficulty getting started, as it is potentially the most ambitious project here.