We have to start with the program behaviors in general. How can a program behave, in general? It can produce output, then terminate, panic, or loop forever; or it can keep producing output indefinitely. The common approach, extensional compiler correctness, says the behavior is a sequence of observable events, such as external calls or I/O.

For something like Solidity, the events are logs, calls, storage writes; the behavior kinds are, roughly:

• success with: a trace of events, some return data, and the leftover gas;
• revert with data, and the leftover gas;
• panic / OOG (different from revert).

The semantic of Solidity language should define a behavior for a valid Solidity program. The semantic of its compilation target, EVM, defines behavior for EVM bytecode. Extensional compiler correctness property says: a compiler is correct if the behavior of EVM bytecode matches the behavior of Solidity program.

If a Solidity program has more than one possible behaviors, the compiler should select one and make it the behavior of compiled EVM bytecode. It is a refinement: we may lose some possible Solidity behaviors, but we don't let new behaviors in.

In a more general case, both source and target languages may allow multiple defined behaviors. Then a compiler is extensionally correct if all behaviors of compiled program are in the set of behaviors of source program. A bit simplified:

\[\forall P, Behaviors_{target\_language}(compiled(P)) \subset Behaviors_{source\_language}(P)\].

A trace of observable events is too simplified to catch all important properties of programs. Say, the updated compiler produces functionally equivalent, but 100x slower programs. It reads like a bug and we will probably call it so – so is this compiler still "correct" in the mundane engineering sense? Another example: the new optimization pass made it so that most of the time the code is running 2x faster, but on rare occasions it lags and stops responding for an hour. Is it acceptable? Now what if the lag is not an hour, just one extra microsecond, but the attacker can exploit it indefinitely and keep your code lagging forever? Finally, imagine you are a mobile developer and after updating your compiler whatever app you build for Android drains the battery in an hour. Is it a compiler bug?

Extensional compiler correctness is oblivious to the resource consumption, including time and power; it does not understand average or worst case; it can not capture the time to respond. It is all about "what" and nothing about "how".

But it gets worse. If the program can inspect the "how", then optimizations change semantics. Imagine a function \(f\) that typically takes 11ms to complete, and the following code.

started = clock()
f();
elapsed = clock() - started;

if (elapsed < 10ms) {
    print "yes"
}
else {
    print "no"
}

We branched on the timer value: if the program has run quicker than 10ms, do one thing, otherwise do another. Now suppose we launch compiler with different optimization options, or we have updated the compiler, or switched to a better one. Function \(f\) is now optimized and runs under 10ms every time. The optimization changes the behavior, not just how but what!

Gas is one of exposed resources, and it is hard to ignore. Of course, we care about compilers producing gas efficient programs that cost less, and we care about gas attacks too. But if a program can also read the gas meter, the same way they could read a timer, they can branch on the gas value, producing different behaviors. Again, the resource is not just affecting "how" the program runs but also "what" it does.

To make matters more complicated, gas is a beast with not one, but three heads. Each head bites the compiler correctness issue from a different side.

Gas is an observable effect. Publish the value of gasleft() in the storage, or introduce gas-dependent control flow – it is now affecting the trace.

Gas is a resource bound. An optimization can make a cold path more expensive to reduce the average cost. Then unoptimized code may complete when provided the bare minimum of gas, but optimized code on the cold path will run out of gas. Most formal specifications of compiler correctness will treat it as a change of behavior, meaning that such compilation is not correct.

Gas is a part of execution environment. A platform upgrade adjusts the cost of instructions, or changes how gas passes through cross-boundary call. Programs that were terminating correctly may now run out of gas. In Ethereum ecosystem this has already happened e.g. EIP-2929 repricing, or EIP-1884.

We can classify programs by how gas affects their observable behavior:

gas-sensitive

behavior depends on exact gas (reading gas meter value, branching on it, putting it to storage etc.). Gas is an observable effect here.

gas-insensitive

changing gas consumption does not change observable trace, unless we run OOG. Gas is just a resource bound here, except for potential OOG that we can observe.

gas-fragile

programs break if we change the protocol parameters such as gas budgets / stipends / EIP-150 gas forwarding 63/64 cap/instruction costs. The way environment treats gas matter here.

Note that these are not properties of a single contract; they are properties of a contract execution, which means they span across the whole call tree. A contract might be gas-insensitive, but once it calls another, gas-fragile contract, the whole thing becomes brittle. In an open ecosystem like Ethereum, this is important, because if you call an upgradable contract, you have to trust that the upgrade won't break your gas assumptions about it. Of course, the callee has to be aware of such assumptions in the first place.

The second problem mirrors the first: compiling the gas-aware language for a target with different gas consumption. How to compile Solidity for gas-fueled RISC-V? Maybe we should treat Solidity gas as some abstract resource, the platform-native gas? Then retrieving the exact gas values should be either prohibited or made into platform-dependent language extension. Or, perhaps, Solidity gas should map precisely to EVM gas, its native counterpart, but on other platforms we will have to simulate it. Then the EVM gas becomes meaningless on other platforms, because they only accept their proper gas as a payment for running code. Programs won't survive depleting neither native nor simulated gas.

We circle back to gasleft() here: its intention seem to reflect an approach "just expose the internals as they are". gasleft() inherits its semantic from EVM and the protocol in general. The meaning is platform-specific and, for the higher-level language, almost accidental. Trying to support another compilation target when Solidity and EVM are tightly coupled together feels like compiling x86 assembly to ARM, while trying to keep stack usage perfectly aligned.

Worse even: in EVM, the cost of an instruction that touches memory depends on the current memory size (roughly a quadratic law). Storing something outside the current memory bound triggers expansion and you pay for every new byte. Move a memory-touching instruction earlier and you might expand memory earlier. Later instructions touching memory will do it at a change cost. If we want a perfect gas virtualization, we have to account for this too. And moving around memory operations for optimization purposes may affect observable behavior through meddling with the gas consumption.

There is another language that likes to expose features from the lower level of abstraction – the good old C. Writing directly to registers, switching CPU modes and so on belong to platform-specific extensions, implemented in compilers supporting the respective platforms. They are not portable by definition, and portable code is decoupled from them. Macro definitions are often used for that. Gas is more pervasive; without a gas definition generic enough to describe EVM, and RISC-V, and other targets, we have to virtualize EVM on other targets.

Any gas-aware language will have to deal with both problems: how to support multiple compilers for the same target, and how to support multiple targets. My intuition is that to solve both of them we need to figure out a formal gas consumption model and guarantees that it can realistically provide. Furthermore, we won't be able to have a singular definition of correctness, because there is a tradeoff between gas consumption predictability and gas efficiency. These compromises might be more or less friendly to the portability of contracts. To advance further, we need to figure out the semantic of gas for the language. Can we do it for Solidity?

Unfortunately, it seems the most popular language to write contracts in Ethereum ecosystem is long past the no-return point. Now, chaos dragons live here.

First, Solidity does not have a complete specification separated from compiler implementation. This means that the default compiler solc is the description of the language. So, if your compiled contract acts weirdly, it is challenging, if even possible, to diagnose whether it is a compiler bug, a bug in the language specification (which does not exist), or a correct, albeit unintuitive, behavior.

Second, other Solidity compilers have to reinvent the language specification by reverse engineering the existing compiler, solc. This includes separating out the compiler bugs. Without a language standard decoupled from solc, "correctness" is just "what the other guy did." But we do not have a choice at this point.

Third, solc is written in C++, a language rich in undefined behaviors, which are sometimes borderline impossible to catch. Therefore, solc itself is likely to exhibit undefined behavior on some inputs. Undefined behavior in the compiler may be contagious to the programs it operates on. This makes using solc as a source of truth for the semantics of Solidity even more dangerous. Tomorrow we recompile solc with a new version of clang or change the arguments in its invocation. After that, it will perhaps compile Solidity programs into the bytecode with diverging behavior.

The Gas Dragon enters the club and finds itself at home. The gas consumption is treated as an implementation detail, the guarantees about it are not documented. Without an explicitly specified interface, the Hyrum's law applies even to smallest user bases.

A theorist in me shouts: do not make your programs dependent on exact gas values! Do not make strong assumptions about gas! Distill a language standard describing "abstract gas" which is guaranteed to satisfy 42 different formal properties; your programs should respect the constraints of the standard, or their behavior won't be well defined. In case of a niche language without an established codebase, used mostly by programmers well versed in CS, perhaps. We can even throw in something to provide static guarantees about gas consumption, maybe linear types, resource monads etc. But can we do that to a language with millions of lines of code already written and deployed? We know how well such retrofitting worked out for the mess of C(++), right. Ask a couple of your fellow C programmers to recite the precise definition of a pointer, then check how far they diverge from the standard. A pointer is not "just a number", and null pointer is not zero (if you think of zero as a number with all cleared bits, not the literal 0 in a pointer context…)

So, I don't think we can universally retrofit some abstract gas semantics to Solidity without dealing with the same problems as early standards of C, which were trying to harmonize some thousand programmers with diverse backgrounds and qualifications, and diverging ideas about pointers, memory, type conversions and so on. But maybe we can contain the damage or at least take lessons because any future gas-aware language will face the problem of gas-dependent observable behaviors. Solidity, because of how popular it is, shows us how real people write contracts, what are the challenges, what do they really want from gas meter.

I believe we still have to investigate the relationship between gas and program behavior deeper. It will most certainly be a semantic parameterized with the protocol version; it may describe several possible tradeoffs between predictable gas consumption and optimizing the costs; it should account for memory expansion costs and make clear the possible reorderings and optimizations. Even if we won't be able to fit any or all of them neatly into Solidity, it will give us ideas about how to design better gas-aware languages. But I'm leaving this for the next time.

In the previous section, we provided an example of a program specification: “output all even numbers in the range from 0 to 100”. If a program outputs a list of numbers without any special properties, it is easy to verify its output – just make sure it contains all the expected numbers. All outputs that contain the required numbers are equivalent when we think about the program correctness. This equivalence is easy to define, and computers can verify it.

Compilers are way more complicated. To decide if a compiler is correct for a certain input program, we need to show that the observable behavior of an input program written in X is equivalent to the observable behavior of the output program, written in Y.

Unlike comparing two lists of numbers, equivalence of program is not just harder – it is undecidable in a general case. Imagine needing to put programs in every imaginable context and see how they behave – there is no algorithm that can do that for all programs, in a finite time. This follows from Rice's theorem.

How we define an observable behavior is situational. For example, if we are interested in how exactly the program is executed, not just its results, we may include the memory allocation events. Otherwise we can completely ignore all operational aspects of a program, reducing it to its output.

CompCert, the first verified C compiler, defines an event in a C program as a system call or read/write to volatile memory; then an observable behavior is either a finite trace of events, or an infinite trace (repeating the same sequence of events or diverging), or an error. This approach can be traced to systems theory:

As we have seen, systems of equations […] may have three different kinds of solution. The system in question may asymptotically attain a stable stationary state with increasing time; it may never attain such state; or there may be periodic oscillations.

— "General Systems Theory", Ludwig von Bertalanfy

Let us simplify the task: instead of verifying the correctness of the compiler for all meaningful programs in X, let's focus on identifying specific bugs in the compiler. While testing it on different programs in X, we may encounter a program that behaves differently after compilation. This could seem like like a strong evidence for a compiler bug. Surprisingly, this is still not sufficient to identify a bug in compiler. But why?

To compare the behaviors of two programs, we need to answer several complicated questions.

1. What is a program’s behavior? This, in turn, requires defining:
   • The language semantic for X;
   • The language semantic for Y;
   • Observable behavior i.e. which events are visible.
2. Can a program in X exhibit multiple behaviors? What about compiled programs in Y?
   • If yes, is it acceptable to lose some of X’s behaviors in the compiled programs?
3. Can a program in X exhibit undefined behavior? Additionally:
   • Can undefined behavior be reliably identified?
   • Does the compiler guarantee anything about the behavior of the compiled program in such cases?

These issues hint to the first major problem: bug identification/definition. Given a compiled program that behaves unexpectedly, what is the underlying cause?

If we are unsure whether we have found a bug or if the unexpected behavior stems from an incomplete language specification, distinguishing one bug from another becomes even more difficult. How can we tell whether two miscompilations result from the same compiler bug or from different ones? This is the bug identity problem.

Why does bug identity matter? Can't we just fix bugs one by one? Sometimes, fixing one bug will inadvertently fix another, suggesting that they have a common root cause.

First, if you have 100 issues in your bug tracker, it helps to know whether any of them are related. This aids in planning, as it helps estimate how many patches do we really need to resolve these 100 issues.

Second, there are multiple ways to incentivize the community to report bugs for some sort of compensation. When two people report the same bug, how do we ensure fair compensation? It is hard to say whether they have reported the same bug at all! All reward strategies seem flawed:

• A "first-come, first-rewarded" approach does not work because new reports may be related to already reported but unresolved bugs.
• Rewarding both reporters equally is problematic because one can spam the system by generating numerous programs triggering the same bug, receiving disproportionately large rewards or reducing the reward for the original reporter if the total reward pool is fixed.

Futhermore, compiler users are often application developers, not experts in programming language semantics or programming language theory. They may not understand why, when they report something that is clearly a bug to them, the compiler experts scratch their heads and come up with an excuse – obviously because they don't want to pay or are ignorant.

Languages like C and C++ are rife of "bugs" of this kind. Many rather arcane behaviors are only recognized by a selected few, well versed in the language standard. For instance, using unions for type punning, where one union field is written and another is read, falls into undefined behavior according to the language standard.

union array {
        char[8] as_chars;
        uint64_t as_uint64;
};


union array my_union;
my_union.as_uint64 = 42;

// accessing the 3rd byte of the variable holding 42.
char n = my_union.as_chars[3];

In this case, the C standard deems such practices as undefined behavior, meaning the developer should not expect consistent behavior. Therefore if a developer finds that the variable n equals 999 in their code, regardless of the value written to my_union.as_uint64, jokes on them – the language standard is on the compiler's side.

One potential way to define bug identity is to say that two bugs are the same if one cannot reasonably be fixed without also fixing the other. This definition is imperfect, as the term "reasonably" is vague. In theory, we could patch specific cases without addressing the root cause, but this is impractical for compilers, as similar bugs will continue to arise.

In the end, bug identity is hard to formalize. Calling in an expert may be the best option, relying on open criteria to judge whether two inputs trigger the same bug or different ones. Some cases are clearly not bugs, while most fall into a gray area.

After distinguishing between two bugs, the next step is to prioritize which one to fix first. This is the bug severity issue.

Compilers are usually tested using a code base containing both synthetic tests and real projects. It makes sense to use the real projects affected by the bug as a part of work prioritization.

However, the identity problem complicates this process: decision-making requires comparing bugs across different input programs. How do we identify duplicates? How do we tell if the same bug is affecting two programs or if two different bugs are at play?

As mentioned earlier, automating bug identification and comparison is not feasible, meaning expert judgment is necessary. Malicious users who understand compiler bugs may submit thousands of inputs that trigger the bug, flooding developers with AI-generated bug reports. This can amount to a real-world denial-of-service attack on the development team. Fairly rewarding bug reports under these circumstances becomes nearly impossible.

Most programming languages lack formal, unambiguous specifications, making it difficult to define correct program behavior and identify bugs in compilers.

Comparing bugs is challenging because they stem from program behavior. Programs may have multiple possible behaviors and the compiler will select one of them. Since program equivalence is not decidable for Turing-complete languages, we can't reliably detect when the compiler's output changes but the behavior remains the same.

In the next post, I will explore the challenges faced by compilers for platforms where computations are paid for with gas, particularly when the compiler has multiple backends with different gas models.

The main components of my system are:

1. Zettelkasten, stored in a git repository and organized in Emacs with org-mode and org-roam.
2. Book storage: cloud storage with digital articles and books, and a bookshelf with physical books.
3. An Android tablet with a stylus to keep a working set of documents and annotate them.
4. An e-ink reader, which mirrors a part of my collection of books.

The main stages of my workflow are:

• Reading/watching and making notes as I go.
• Revising, copying notes to my long-term note storage.

Now, to the details.

Most of the time, I study by reading books, articles, and blog posts. Watching videos is less convenient, but sometimes I have no choice. I will illustrate the workflow on an example of a physical book.

Most of my books or articles are in digital format, so I read a lot on my tablet as well. For that, I have bought Samsung Galaxy Tab7 Fan Edition this summer, and it is just really good for me:

• The screen is big (and I never felt like I have enough screen space for comfortable handwriting before)
• Samsung S-Pen feels good to me, better than Apple pencil. The feel is a bit closer to the fountain pen, which is my writing tool of preference on paper.
• The performance is adequate for everything I need.

As for software, I found that Flexcil works best for me. It does not choke when I open 40 tabs with documents, stays responsive and handles even huge books like Differential geometry reconstructed, which has around 2500 pages as of today.

The editing capabilities are not great, e.g. you can not rotate your handwritten text, and the Android version lacks a good synchronization mechanism – you basically have to do backups and restore them. The backup process is fast thought, so I am just doing backing up my notes in cloud every day. Once these features get implemented, Flexcil will become an ideal tool for me.

I also read for pleasure, without annotating. For that I still prefer using my old e-ink reader PocketBook 902 E-ink displays are easier on my eyes. It has now turned 12 years old, and still works like a charm.

At some point I was considering buying an e-ink tablet for annotating, but I find that having color on annotations is too valuable. The current e-ink tablets were not convincing to me.

• Do not try to create a hierarchy (taxonomy) for your notes. It is not scalable and only works for narrow domain. Prefer tags.
• I did not perceive this system as too complicated, until I have written this post. Start small, build a habit, grow as you see fit.
• Without a habit, your Zettelkasten will stay empty. You may need to force yourself to stop reading at some point, revise your notes and put them to the permanent database. Do not delay this too much, or you will not be recalling the annotated texts, but rather reading them anew.
• Redundancy is good. You will naturally put the same information in many notes, and it is all right.
• Drawing is great. Emacs and org notes support .svg files, but you can also include photos or screenshots of your hand-drawn diagrams, why not.

• Think carefully which software you will be using, if any. It is great to have the second brain for life. My first digital notes were in some note-taking app for Symbian OS, then in Evernote. These programs are proprietary and do not always have an easy way for you to migrate your notes somewhere else. Emacs (and text files) is clearly not everyone's darling, but it is free, open-source, and has been around for more, than I am alive. It means that it is flexible enough to adapt to the new technological context: we are not using the same computers with the same operating systems as in 80s anymore, and emacs runs fine on basically everything. I am quite certain that:

  • either I will be able to keep my notes there for life,
  • or I will be able to migrate them somewhere, should such need arise.

  

  Figure 1: My favorite pen since my teen years is Sheaffer Fashion.

In our lives we encounter multiple collections of objects that we call systems. System is a collection of components interacting with each other and outside world, giving the system additional properties, that individual components do not possess. Such properties are called systemic or emergent.

This drawing shows a system of three components, denoted A, B, and C. Component B interacts with A and C, while A and C do not interact directly with each other. A big circle is separating the system from the outside world.

The world around us is filled with systems. Here are some examples:

1. Cars can transport passengers and objects. This ability is a systemic property of cars. No part of a car can transport objects like a whole functioning car.

   Moreover, when the car is assembled but not running, it stays still. Only the interaction between the parts of a car allows it to be a transport.

2. Living humans are systems. Unlike in a corpse, our cells are interacting. Our ability to walk, think, breathe, being alive and conscious is our systemic property.
3. Computers will not perform any work unless powered up and turned on. The ability to compute is their systemic property. It is born from interactions of software and CPU, memory, storage, buses, and other hardware components.
4. A team of software, or hardware developers is unable to produce a complex piece of software unless all participants interact with each other, making sure that they are on the same page. The ability to produce an output together through solo work and interactions is a systemic property.

In this part we will continuously revisit these three examples: cars, humans, computers, and teams.

There is a saying: "a system is more than a sum of its parts." Indeed, a system is not only a sum of its parts, but also of interactions between them. Disregarding the interactions takes the emergent properties out of the picture.

Systems emerge in many areas of our lives, e.g., sociology, biology, physics, economics, computer science, and engineering. Systems theory has become an interdisciplinary field studying systems models at a high level of abstraction, disregarding specific domains.

Systems do not exist in the void, they are interacting with the world.

1. Cars interact with the driver and the road, the trees around them, other cars.
2. Humans interact with other humans, sounds, visuals, the air we breathe, and the food we consume.
3. Computers interact with other devices, with humans through screens, keyboards, and other interfaces.

Environment is a collection of objects outside the system with which the system engages in meaningful interactions.

The context of our analysis dictates which objects should be part of the environment. The car might, in some cases, interact with blue whales, sound waves, bacteria, or asteroids, but most analyses can safely ignore them. If the system interacts with an object from an outside world, we include this object in the environment. The interaction should be essential for our analysis.

The environment usually includes an organized part and a chaotic part. Sometimes it may be viewed as a system on its own, consisting of multiple systems and interactions between them. If we design our own system, we often want to protect it from interacting with the chaotic part and set up the correct interactions with the organized part.

Usually only a handful of the system's components are interacting with the environment directly, and we call them the interface of the system. Here are some examples:

1. The interface of cars includes its controls and wheels.
2. The interface of humans includes their eyes, ears, and skin.
3. The interface of computers includes external devices, e.g. keyboard, screens, mice, network adapters.

In other words, interface consists of such components that can be viewed both as parts of the system, or of its environment.

The interface can vary depending on the system's function: if we consider a verbal conversation between two humans, their interface is a spoken word; however, when these humans dance together, the interfacing happens through visual contact and touch.

An interface is like a border between the system and its environment. The border consists of such parts of the system that can be attributed to either the system itself or its environment. It is not always easy to define the border well, it may be blurred.

In computer systems, we peek through the interface of a computer system and observe the processes inside. Then we interpret our observations, and this interpretation is what constitutes computations.

The system of interest may be very complex. It may be difficult to decompose, i.e., identify its parts. There are several distinct ways of doing it.

Imagine we have several programs running in parallel on the same computer. Each program is a component of this computer system.

If these programs are threads of the same process, they are sharing the same address space. Therefore, one program can affect the functioning of another by corrupting its memory structures, either deliberately or by mistake. On the other hand, if the each program is running in a separate process, it has its own virtual address space. Then an error in one process will not influence the memory of another process, provided they are not engaged in any other interactions. There are of course details to that, like what happens when a process crashes while another one is waiting for its response, but such situations can be dealt with in a reasonable way e.g. through timeouts.

There is a concept of modularity in system design. It means that to construct a system we start with smaller building blocks, which are built in isolation. Then we assemble the bigger system from these blocks, setting up connections between them.

In the example, we see two degrees of modularity. When programs share their address space, the modularity is softer, it does not prevent unintended interactions from happening. An error in one program (module) can then affect other programs, disrupt their functioning and lead to more errors. This error propagation is out of control and leads to unpredictable consequences.

On the other hand, programs in different processes are less fragile. Then the modularity is stronger.

To sum it up:

weak, soft modularity

errors inside one module can propagate to other modules bypassing interfaces.

strong modularity

modules are more isolated and the error propagation is limited to the interfaces.

These two notions are pure and extreme, real systems often fall somewhere in between.

Tail call happens when the subroutine ends with a call to another subroutine. In the higher level language this happens in cases such as:

void g();

void f() {
    g();
    return;
}

This is also a tail call, because we just return what another function returns:

int g(int x);

int f() {
    return g(42);
}

This is not a tail call: after calling function we have to multiply its result to another number. We wait until fact(n-1) completes its execution, and then use its result in other computations. Note that in this example the function calls itself rather than some other function, but this is unimportant.

int fact( int n ) {
  if (n < 2) return 1;
  return n * fact(n-1);
}

On the assembly level, a tail call corresponds to the following pattern of instructions:

f:
  ...
  call other
  ret

This pair of instructions is located in a function which we are going to call f. The control reaches this function from some other function, say, f_caller.

Let us follow the state of stack through the execution of f_caller, f and other. For that we expand this code to include f_caller and other functions.

f_caller:

...
   call f
   <next instruction> ; mark 4
...

f:            ; mark 1
  ...
  call other
  ret         ; mark 3
...
other:        ; mark 2
   ...
   ret

• When we start executing f the stack holds the return address inside f_caller (we reach mark 1).
• When we call other the stack holds the return addresses for f_caller, then f on top (we reach mark 2).
• The subroutine other returns too, in this moment we have only return address for f_caller on top of the stack (we reach mark 3).
• The subroutine f returns, popping return address of f_caller from the stack (we reach mark 4).

The last two steps were essentially popping two return addresses from stack consecutively. It suggests that the first one (the return address to f) is useless and we do not need to store it. We are indeed right.

The call other instruction is equivalent to the pair of pseudoinstructions:

push rip    ; rip is the program counter register
jmp other

If we do not need to push return address to f, then we can just substitute this instruction for jmp and get rid of one ret:

f_caller:

...
   call f
   <next instruction>
...

f:
  ...
  jmp other
...
other:
   ...
   ret     ; will return directly to f_caller

The subroutine other becomes essentially the continuation of the subroutine f; we pass from f to other via a simple branching instruction.

Coming back to our example, we can rewrite it like that:

print_char:
   ...
   ret

print_newline:
    mov rdi, 0xA          ; code
    jmp print_char

Our current example is the implementation of print_newline in two instructions:

print_char:
   ...
   ret

print_newline:
    mov rdi, 0xA          ; code
    jmp print_char

When the example is written like this it is easy to guess the next step. If the execution is sequential why bother with a jump here? The control routinely falls from one instruction to the next one anyway.

print_newline:
    mov rdi, 0xA          ; code
print_char:
   ...
   ret

The subroutines have merged here, and it does not look like there are two separate functions anymore. We got a subroutine with two entry points labeled print_newline and print_char. We call such a thing a coroutine.

Coroutines are a generalization of subroutines, they may have a state and multiple entry points. You may often encounter them or their variations in modern languages e.g. generators in Python or lazy IEnumerable's in C#. Our coroutine does not have a state, however. It is funny to note that coroutines appeared first as a assembly programming pattern and is not a fancy new thing like cubical type theory provers.

If your language supports continuations, you can implement coroutines easily.

I do like to draw connections between seemingly distant notions and areas. I also enjoy learning new points of view on things I already know. My students inspired me to write this post because for a lot of them, coming from higher level languages, the functions were entities with well shaped bodies, having a visible start and end. Well, in assembly they do not have to.

It also supports a way of thinking about compiled code as a "soup" of sorts, where everything is blended together, reordered, precomputed, optimized until only the actual actions and their order suggest how the original program looked like. I think this is especially useful when reasoning about parallel programs and lock-free algorithms.

• Eating well is important. Proteins and fats are good for breakfast.
• You have a daily limit of several hours of productive work. This is why working every day is so important.
• Try to concentrate for 15 minutes straight, if you succeed the concentration will persist.
• If you have writer's block, read. It gives food for thought.
• Separate writing (create) from editing (reduce, arrange).
• There are rules for writing, which work most of the time. They are dictated by empirical evidence on providing the best reading experience and by the experience of other writers.
• First draft should be longer than the final version (probably 25%).
• Better grab the reader's attention immediately, so do not forward reference in the beginning.
• Write often to structure your thoughts.

There are many choices we make when we write. These choices affect the text semantics. We can categorize them based on the "level of resolution":

• Selection of words.
• Sentence structure.
• Order of sentences in a paragraph.
• Order of paragraphs in an essay.
• The essay as a whole.
• Essay as seen by a reader (prism of his mind and personal experience).
• Essay in a context of culture the reader is in.

1. Choose a topic
2. Make a reading list and read; take notes (probably 2-3 times the amount of final text you need).
3. Write an outline (10-15 sentences). Stock introductions/conclusions are ok, but should be thrown away after.
4. Write a paragraph per outline heading (10-15 sentences). Do not edit too much. A paragraph should present a single idea.
5. For a single paragraph: place each sentence on its own line. Rewrite each sentence to make it smoother and better. You can read it aloud and listen to yourself. Try to cut length by 15-25%. Do for all paragraphs.
6. Repeat by looking at each paragraph as a whole. Sentences that are no longer necessary are to be eliminated here.
7. Read your essay. Try to make an outline without looking at the text. It will help throwing some sentences away, rearrange them etc.
8. Wait a few days and re-read everything. When you are rewriting something but you are not sure it becomes better anymore, it is time to stop.

In order for variance to even exist we need the language to have the following features:

• It has to be typed

• It should sometimes allow an entity of type A to be implicitly interpreted as an entity of type B.

  The most common cases are:

  • Subtyping in class hierarchies.

    Basically, everything that has an "Object Oriented Programming" label on it: C++, C#, Java etc.

  • Implicit type conversions (coercions).

    For example, in C/C++/Java, there are implicit conversions from numeric types into their wider versions, like short to long, or float to double.

• We also need parameterized types.

  It does not mean a presence of generics though, as functions alone are enough to introduce covariance and contravariance.

In other words, we need to be able to represent types as an oriented graph.

Here I want to mention a couple of facts it is useful to be aware of.

You certainly do not need to be familiar with formal logic or categories if you want to just know one practical thing (say, frontend) and only do it. There are two main paths which differ by an effort, duration and outcome.

• You can become proficient in one domain relatively fast — say one, two years. You will not be useless, you will do things and make a living. There are enough job opportunities (at least, for now) which do not demand much versatility.
• You can become a well rounded specialist who invested a lot of time and effort in foundations. Then you will be able to adapt and switching career paths becomes relatively easy. You can do machine learning, then formal verification, than some low-level programming for trading or switch to game dev. That demands time and dedication — I’d estimate a minimum of 6–8 years.

I strongly advocate the second path because it’s more versatile, interesting, and brings more in the long run. IT is ever-changing so you want to pick up new technologies fast. You also have more choice. Should you choose the hard way, the rest of this post should be of a use for you.

I can not stress that enough. When you start, you might think that you don’t need linear algebra, because you are unaware of applications. However, for any non-trivial machine learning, you need it. You need statistics and probability. You need logic, combinatorics, set theory, all sorts of discrete mathematics, graph theory, computability, formal grammars, lambda calculus, formal semantics, topology, type theories, a bit of number theory, groups, rings, fields, categories.

New technologies are constantly emerging. Many of them are based on the existing mathematical models. If you know the underlying mathematics well, you get very nice perks:

• Picking new trendy things is orders of magnitude simpler.
• You understand where you can apply new methods and where you should not.
• You usually understand why the solutions are the way they are. Then you can tweak them to better suit the context.

For example, I have had an impression, that few people understand, that you should not always use least squares to evaluate how well your linear regression fits the data. This is only adequate when the errors are distributed normally with the appropriate mean value. If it is not the case, you will blindly apply an inadequate solution without even thinking that a part of the model needs tweaking.

Writing proofs makes you rigorous. You want to always think about all possible paths of execution your program can take in order to not introduce bugs and security issues. The clarity of thinking gained from constructing proofs is precious. It also helps you in writing short, concise code.

It should be well designed, which means:

• Consistency.
• Small core.
• No unnecessary complexity (it often comes from inconsistency: there are things you should just remember or constantly be aware of, that bring nothing useful to the table).
• Makes it harder to shoot yourself in the foot.
• This should also be a high level language, because programming is problem solving, not a mastery of a specific language. Knowing all little particularities of your favorite language is not a mastery of programming in itself.

I advise one of these languages:

• Scheme (there is a good classical introductory course “Structure and Interpretation of Computer Programs”)
• Smalltalk
• Eiffel
• ML

Don’t be fooled by their seeming unpopularity, in the programming world the popularity does not mean quality. Do not start with Python, pretty please! It is badly designed, inconsistent, and does not teach you rigorous thinking. No need to get used to "well, it seems to usually work" mentality. Python has its uses, but not as a first language.

If you get used to crap languages and crap tools, and crap software, and crap solutions, you will inevitably replicate them in your own work. Be critical, question everything, critic everything, search for inconsistencies and flaws.

For example, imagine you are learning a new language, Go. Google "Go language sucks" and read why people criticize it. Some of them will be pathetic, but some will actually have a point. It is likely, that you will obtain new knowledge from reading critical remarks and evaluating, whether they actually have a point, or are just there to whine.

I am teaching programming (C and assembly) since 2009 to the students in ITMO university in St.Petersburg, Russia. A lot of people have trouble programming and never actually succeed in learning it because of inability of creating code. When they get an assignment, they try to imitate an existing solution, maybe take some snippets from Stack Overflow, tune them to their liking. OK fine, you got your solution, what else do you want?

You should learn to write code from scratch. The types of skills needed for that are so different from meddling with existing code!

Programming is about making conscientious choices. You are in state A (you have access to a number of language features/libraries and you know how to combine them); you want to get to state B (the language constructions are combined in a way to express a solution). How do you build a route from A to B? Now, that is the real programming, the problem solving.

When you start writing programs from scratch, it will be hard, but it is absolutely necessary to learn to build things from zero. To improve your problem solving skills it is crucial that you learn algorithms and data structures. Pick up a good book and solve contests online. I recommend “Algorithms” of Dasgupta for a start, then the classic work of Cormen. This will open a whole new world for you, I promise.

The complimentary part of software creation process is designing the software architecture; it is impossible to learn to structure your programs well without building them from 0 to 100.

Program everyday, do side projects. There is a very easy (and mostly accurate) way for me as a teacher to understand that my student will succeed as a programmer with a high probability. One question: What are you programming in your free time?

There is just not enough time for your teachers to tell you about everything. After all, after you are out of the university, you have to continue to learn on your own, until you retire. If you are passionate about what you are doing, you will explore different types of software just for fun, and that will give you much more experience and skills, than your less motivated peers will have.

Ideally, you should touch everything: write your own compiler, maybe a toy OS, http server, database engine, games, ray casting, build some neural networks, fiddle with proof assistants and dependent types, write a simple mobile app, write for embedded … you go on. Place all your projects on GitHub and take pride in them: your future employer might have a look at it. Use this portfolio to your advantage.

It is common knowledge, that recruiting a good programmer is extremely hard. Many programmers applying for jobs have trouble writing trivial things like FizzBuzz. If you have existing projects hosted on GitHub, the employer will be more assured that you are the real deal.

If someone tells you all languages are alike, this is either an oversimplification or a lack of experience. Let me explain that a bit.

A model of computation is a set of basic operations and ways of gluing them together in order to build complex algorithms. Some languages have very similar models of computations, but some are very different.

Programming is so much bigger than your commonly known C/Python/Java/C++/C#/Go/Javascript, which are all built on the same principles: imperative, structural, with occasional bits of OOP and syntactic sugar to mimic other programming styles. The world of programming languages is huge, here is a little taste of it:

• Industrial functional programming languages with complex and well thought out type systems (Haskell, Ocaml)
• Functional languages with dependent types, which allow not only to program, but to write proofs of correctness (Coq, Agda, LEAN)
• Stack based, concatenative languages? (Forth)
• Logic programming (Prolog, Refal)
• Finite state machines (regular expressions, Promela)
• Heavily extensible languages allowing to implement virtually any syntax constructs, as Lisp, Forth, Camlp4/5 or Rebol allow.
• Domain-Specific Language workbenches such as JetBrains MPS or XText

Every new model of computations is hard to learn, because it is a new way of thinking for you. But the investment is worth your time, because once you get familiar with it:

• Every language based on it is easy.
• Every language whose model of computations has similarities is easier.
• As every model of computations is very fitting for specific type of problems to solve, you now have a new powerful tool, whose usage in specific contexts is orders of magnitude more productive.

I have been very fortunate to know some amazing people. My mates helped me to perfect my skills, to learn something new, to see the world from a different point of view. Isolating yourself will bring you no good in the long run: you need other people to discuss, to see what they are up to, what they think. If your mate has read an interesting article and told you about it, you just saved a lot of your own time, because he spoon-fed you with a processed, crystallized knowledge.

You will be surprised, how much you can learn during a lunch time with your mates, who are eager to share the details of their work or research. This kind of idea cross-pollination is one of the main reasons corporations like Google give you free quality food.

Looking at someone’s work, given he is better than you, can teach you a lot, in ways you do not expect. Code reviews are even better, because the guy will tell you, how he would have written the same code. This is probably one of the most effective ways to become a better coder very, very fast.

This is so important, that it has a section on its own. Tests are an integral part of creating software, and even guys like me who are working on formally verified software (which means it should be mathematically proven correct) are writing tests, albeit one might think, that the guarantees given by proofs are strictly stronger.

I hope that this might actually help someone to get a bigger picture, learn faster and become a better programmer; should you have any questions, I will be glad to help. Good luck!

Suppose we are in a metric space with Euclidean metric. The space is a sphere itself, with radius \(r\). Let us denote a closed sphere of radius \(r\) with center at 0 as \(D_r(0)\). This sphere is a subset of \(R^n\).

What is a sphere? By definition it is a point of a space and a set of such points that are no further than the sphere's radius. What if we try to define a sphere of radius \(R > r\) inside the space we are working in?

Why does this sphere look as an intersection of two? We are, by definition, effectively selecting a subset of points of space \(D_r(0)\) whose distance to \(B\) is less or equal to \(r\). Any point of \(R^n\) lying outside of \(D_r(0)\) is out of reach.

As long as \(R < 2r\), we can fit a bigger sphere inside of a smaller one. Once we cross the \(2r\) border, even if we pick a center \(B\) on the border of the \(D_r(0)\), the longest distance between two points inside of \(D_r(0)\) can not surpass \(2r\). Thus, all points will be contained inside of \(D_R(B)\).

We can also pick up a less traditional kind of space, for example, a graph. Let us label its edges with numbers and take the shortest path as a metric.

It is easy to see that \(D_{1.5}(D) \subset D_{1}(C)\), for:

\[D_{1.5}(D) = \{ D, A, C\}\]

\[D_{1}(C) = \{C, D, A, E\}\]

Radix tree is a data structure to implement partial mapping from integers. It is extensively used to implement all sorts of maps, such as:

• Memory: maps block IDs into block contents
• Memory block: maps offsets into block elements.
• Symbol table: maps global symbol IDs into memory blocks.
• Environment: maps local symbol IDs into pairs of block ID and symbol type.

The type is defined in CompCert in lib.Maps.PTree:

Inductive tree (A : Type) : Type :=
   | Leaf : tree A 
   | Node : tree A -> option A -> tree A -> tree A

A node with no children (a leaf in terms of trees) is encoded as Node Leaf val Leaf. Nodes can store values or be empty. The latter is useful because in Radix trees the paths themselves matter.

Positive integers are represented through their binary encoding:

Inductive positive : Set :=
      | xI : positive -> positive 
      | xO : positive -> positive  
      | xH : positive

Since positive integers always start with leading one, we are using xH to encode it. Applying xO appends zero, applying xI appends one. The result looks like the binary representation of a number written from right to left.

For example, an integer number 11 is represented as 1011 in binary form. The corresponding term in Coq will be:

xI (xI (xO xH) )

Radix trees are used to encode partial maps from positive integers into some domain. The domain values are stored into nodes; the path to the node corresponds to its index.

Let us encode a map:

\[\begin{cases}% 1 \mapsto \text{"one"}\\ 3 \mapsto \text{"three"}\\ 4 \mapsto \text{"four"} \end{cases}\]

As a tree, it will look like this:

As we see, the elements are enumerated according to the breadth-first search order.

• PTree is a type of a Radix/Patricia tree; PTree.get returns None if no element has a given index.
• PMap is a pair of PTree and a default value returned by PMap.get instead of None
• ZMap is PMap where any integer can be used instead of only positive ones. It is done through a bijection between all integers and positive integers.

Memory is a mapping of addresses into block contents coupled with permissions map and some constraints.

*Block contents/ maps offsets (integers) into memory values.

All pointers are pairs of block index and an offset inside a block, so it is impossible to jump from one block into another by changing the offset value. In other words, blocks can not overlap by design.

Values are encoded as follows:

Inductive val : Type :=
    Vundef  : val
  | Vint    : Int.int        -> val
  | Vlong   : Int64.int      -> val
  | Vfloat  : Floats.float   -> val
  | Vsingle : Floats.float32 -> val
  | Vptr    : block          -> Int.int -> val

Memory value is defined as follows:

Inductive memval : Type :=
    Undef    : memval
  | Byte     : int   -> memval
  | Fragment : val   -> quantity -> nat -> memval

Such memory values are assigned to addresses, which means that a block of \(n\) bytes holds \(n\) such values.

• Undef is used to mark the cell as uninitialized. All reads involving such

cells are resulted in Vundef value returned.

• Byte is a concrete 8-bit integer. It is also a type of raw memory. Such raw

values can be taken in a pack and decoded in an architecture-dependent way using decode_val.

• Fragment is a usual case of storing data. It contains an opaque value, a quantity and an index, showing where exactly are we in this value.

Additional arguments include:

• A quantity is either Q32 or Q64. 8-byte values have quantity set to Q64, the other ones are Q32.
• An offset in ranges 0..3 or 0..7. As each element of a block represents a single byte, we are storing as many consecutive Fragments as there are bytes in a value. Each Fragment stores its index w.r.t. the value's beginning address.

Note The source [1] states that only pointers are stored inside fragments, while other values are split into bytes according to the architecture specification. The lemmas however never impose such restriction, making cases like Fragment (Vint 4%Z) _ _ possible by construction (and appear in proofs).

For example, executing this assignment:

int32_t x;
int32_t* px;
... 

px = &x

results in the following values being written into memory (assuming x inhabits in the block #4 and a pointer is 32 bits wide).

(Fragment ((Vptr 4 0) Q32 0) ::
(Fragment ((Vptr 4 0) Q32 1) ::
(Fragment ((Vptr 4 0) Q32 2) ::
(Fragment ((Vptr 4 0) Q32 3) :: nil

Memory is represented as a record:

Record mem' : Type := mkmem
  { mem_contents : PMap.t (ZMap.t Memdata.memval);
    mem_access   : PMap.t (Z -> perm_kind -> option permission);
    nextblock    : Values.block;
    access_max   : forall (b : positive) (ofs : Z),
                   perm_order'' (PMap.get b mem_access ofs Max)
                   (PMap.get b mem_access ofs Memtype.Cur);
    nextblock_noaccess : forall (b : positive) (ofs : Z) (k : perm_kind),
                         ~ Plt b nextblock -> PMap.get b mem_access ofs k = None;
    contents_default : forall b : positive,
                       fst (PMap.get b mem_contents) = Undef }

All mappings are implemented as Patricia Trees.

• mem_contents maps block IDs (positive integers) into block contents. It is a map implemented on top of a Radix tree. Block contents are maps from offsets (integers, possibly negative) into block elements.
• mem_access maps block IDs into functions, which accept an offset, a permission type (Max or Cur) and return the actual permissions for this block.
• nextblock is the maximal block ID. All blocks with ids in range from 1 inclusive to nextblock exclusive should exist.
• access_max encodes the following property: for all blocks their maximal permissions are higher than their current permissions.
• nextblock_noaccess encodes the following property: no block has an ID greater or equal to nextblock
• contents_default encodes the following property: for all blocks the default memory cell value is Undef.

1. "Program Logic for Certified Compilers". Chapter 32 "The CompCert memory model"

Magritte's style differs from his fellow surrealists such as Dali or Ernst. I've often got an impression that his approach is quite blunt, a bit like if he threw objects into my face:

• The objects on his pictures are big, take a lot of space on canvas. Moreover, the background is empty and not detailed.
• The palette has usually few colors. Speaking of modern times, it is the same kind of restriction that exists in pixel art, and produces a similar effect on me, that I rather like.
• The colors are rarely bright, which, in conjunction with the previous point, makes paintings look surreal, mystic and alien.
• The shadows and shapes are nicely detailed which makes for a highly contrast image (although you won't see many sharp edges).

Magritte's approach is also quite intellectual because he wants to mess with the viewers and make them think (almost a quote of his). Each pair of a picture and its name is a puzzle to solve. The reward is a better comprehension the message of a picture, which in turn can bring you some new understanding about the real world. The name and the picture can be connected in a number of ways, for example:

• The name can be a hint to solve the riddle. The picture named "Explanation" depicts two bottles, one of which looks like half-bottle, half-carrot. As we see the image and its title, we come to an understanding of what an explanation is, its idea: the explanation is a process of changing our perception and/or understanding of concepts. This change is the transformation of an image of a bottle into an image of a carrot.

  

• The name can be just something vaguely connected to what's on the painting. For example, the name "Imp of the Perverse" should make you think that something wrong is going on, make you uncomfortable, which, in turn, strengthens the feeling of wrongness coming from the piece of art itself. In a sense, Magritte wants you to think critically, not make you trying to connect distantly related terms. We can find a context to associate any pair of words if we want to, though not all contexts are of importance to us.

  

• The name might hold an allusion to other works of art Magritte associated his painting with. For example, "The Man from the Sea" refers to the film of Marcel L'Herbier, which tells a story of a lone sailor. More importantly, it resembles the final scene of "Juve against Fantômas" where this villain in black prevails by setting off an explosion with a pull of a lever.

  

What's especially interesting about Magritte's philosophy is his reflections on images, words and real world objects as well as the relations of the said three worlds. He published his famous text "Words and images" in 1929 in the journal La révolution surréaliste as a result to his extensive experimentation in the late 1920's.

This text studies how words (e.g. poetry) and paintings as the means of expressing ideas. Here is the text in English with a little commentary (it corresponds to the images from up to down, from left to right):

• An object and its name are not inseparable. We can find a better name.

• Some objects have no name.

  Words are parts of languages, and there is no a language which includes all notions from any other one. Take Dostoevsky's "nadryv" or dozens of words to describe different shades of snow in the local languages of northern tribes. We create names based on our everyday needs and what seems important to us.

• A word can be used to refer you to itself.

  This thought is illustrated by a word "sky" inside a closed curve, depicting the said sky. As the label "sky" is placed directly on the image of sky, it refers to this exact instance of "sky", which makes for a funny wordplay.

• Sometimes a name of an object meets the object depiction.

  This is illustrated with a labeled image of a forest.

• Sometimes, a name is used in place of an image.

  The picture shows a stub silhouette of an object labeled with its name.

• In reality, a word can be used instead of an object.

• A word can be substituted by an image.

  A pictogram of a sun is used to substitute the word "soleil" (fr. sun).

• An object can suggest that there are other objects behind.
• Everything suggests that there is little common between an object and its representation.
• The words used to represent two objects do not suggest what is different between them.
• On a painting, words are like images.

• The images and words are seen in an unusual way when on a canvas.

  The illustration shows how a written word "montagne" (fr. mountain) blends in and becomes a part of the object texture.

• An arbitrary figure can replace an image of an object.

  I am not sure what this means, because the words (labels) are not mentioned. However the illustration has all these random objects labeled as the sun. When the object is labeled (possibly with an unexpected word), that's one thing, but when we deduce that the object is replacing sun just because he is in the context (position, effects etc.) we are used to see the sun in, that's entirely other thing.

• The object is never the same thing as its image and/or its name.

  I can not stress this enough, as this is quite a useful piece of the puzzle. I will dedicate the next section to this.

• Visible contours of the object form a mosaic in reality.

  This is a beautiful thing to notice. I can deduce, that every point of the space can be named based on it being a part of some part of this mosaic.

• The vague shapes have the same significance as the well defined ones.
• Sometimes the labels define precise things, and the images are not.
• Sometimes, it is vice versa.

In this section I want to explore the following thought of Magritte: The object is never the same thing as its image and/or its name.

Structured programming is a programming style that (as each style) implies a set of rules and constraints to live by. This style implies an extensive usage of subroutines, loops and code blocks (statement sequences between braces). It usually goes in par with imperativeness, when each programming statement is executed sequentially, and mutable addressable memory – roughly following the von Neumann model of computations. It is safe to say that one of its most devoted supporters was Edsger W. Dijkstra. Object-oriented programming can often be viewed as a slightly tweaked version of it. A large slice of modern programming is structured imperative programming or its variations; moreover, it is what kids are taught in school. Because of that educational system flaw, most of us are deeply infected with an imperative thinking and are often emulating other paradigms on top of it in our heads.

Dijkstra considered goto harmful because it makes harder to follow the code logic. This is usually true, but needs a clarification. What makes goto harmful is that it is usually paired with assignments.

Assignments are changing the abstract machine state. When reasoning about a typical program, we are usually tracing its execution and mark how the values of the variables are changing. Throwing goto ’s everywhere makes it notably harder to follow the program state, because you can jump anywhere, from anywhere. That makes the trace much harder to untangle.

However, throw away the state changes and you will have no problems using multiple goto’s in an isolated piece of code (e.g. inside a particular function), because the interesting state will be determined solely by your current position in code!

If we want to implement a Finite State Machine (FSM), then goto ’s are the way to go! Such an abstract machine consists of:

• a set of states (C labels). One state is marked as an initial state.
• input (sequence of global events, f.e., character input, received network packets, any user actions)
• output (sequence of global actions, responses of the system: send packets or control signals to the connected hardware, output etc.)
• for each state, a set of rules to jump to other states based on the current input.

We start in the initial state and perform jumps between states based on the current input value. As you see, this machine has no memory. If we are implementing them in a language such as C, its state will be characterized solely by the position in the code we are currently at.

Crafting a FSM is equivalent to crafting an algorithm to solve a problem. They are not expressive enough to solve all problems Turing-machines can digest. Nevertheless they are not only potent, but very convenient for some tasks such as template matching in strings, implementing network protocols and robot controlling tasks.

Here is a toy example, taken from [my book](http://www.apress.com/us/book/9781484224021). This FSM checks whether the input string containing only characters 0 and 1 contains an even number of ones. It is common to draw cool looking diagrams for FSM, showing states as circles and transitions as arrows between them.

Let us take a look at its implementation in C. I have omitted error checks for brevity.

  #include <stddef.h>
  #include <stdio.h>

  /* str should only contain 0 and 1 */
  int even_ones( char const* str ) {
    size_t i = 0;
    char input;
   _even:
    input = str[i++];
    if (input == '1') goto _odd;
    if (input == '0') goto _even;
    /* end of string -- null terminator */
    return 1;
   _odd:
    input = str[i++];
    if (input == '1') goto _even;
    if (input == '0') goto _odd;
    /* end of string -- null terminator */
    return 0;
  }

  void test( const char* str ) {
    printf("%s\n", even_ones( str ) ? "yes" : "no" );
  }

  int main(void) {
    test("0101011");
    test("");
    test("010");
    test("110");

    return 0;
  }

An additional benefit is that there exist a quite powerful verification technique called model checking which allows you to reason about the program properties if the said program is encoded as a finite state machine. You can reason about them using temporal logic, checking properties such as "If I got into the state A, I will never reach the state B from there". The model checkers can often generate C program automatically from a FSM description.

For examples of what model checking tools are capable of, I recommend you this exercise page for SPIN model checker.

C++ has a nice feature C lacks. It can automatically call object destructors whenever the object's lifetime is over. For example, in the following code the destructor for an object myC will be automatically called after we reach the closing bracket. But it gets better: every time you are writing a return statement, everything that exists in the surrent stack frame gets automatically destroyed in the correct order (reversed initialization order). Consider this function, which returns the error code and uses three objects: myA, myB and myC. The respective classes should have defined destructors which free all associated resources.

  #include <iostream>

  int f() {
    A myA;
    B myB;
    C myC;

    if (! myA.init() ) return 0;
    if (! myB.init() ) return 0; // myA's destructor is called
    if (! myC.init() ) return 0; // myA and myB's destructors are called

    //...

    // Destructors for myA, myB, myC will be called here anyway
    return 1;
  }

In C we often want to do the same thing, but we do not have that luxury of automatically calling anything. It is, however, very important do to because some structures have dynamically allocated fields or are associated with other resources such as file descriptors. It can easily leak resources. So, to do things right, we have to produce quite a mess:

  int f() {
    struct sa a;
    struct sb b;
    struct sc c;

    if ( ! sa_init( &a ) ) return 0;
    if ( ! sb_init( &b ) ) { sa_deinit( &a ); return 0; }
    if ( ! sc_init( &c ) ) { sb_deinit( &b ); sa_deinit( &a ); return 0; }


    return 1;
  }

Imagine you had 5 structures to work with, this straightforward approach is going to turn your code into nightmare! However, with the help of goto's we are going to exploit a nice little trick. It bases on the fact that all such branches can be ordered by inclusion: each branch looks exactly like some other branch preceded by an additional deinit:

  //
  return 0;
  sa_deinit( &a ); return 0;
  sb_deinit( &b ); sa_deinit( &a ); return 0;

If we throw labels in between we could jump to any statement in this sequence. Then all following statements will be executed as well. This way we are going to refactor the example above to look like this:

  //
  int f() {
    struct sa a;
    struct sb b;
    struct sc c;

    if ( ! sa_init( &a ) ) goto fail;
    if ( ! sb_init( &b ) ) goto fail_b;
    if ( ! sc_init( &c ) ) goto fail_c;

    return 1;

   fail_c:
    sb_deinit( &b );
   fail_b:
    sa_deinit( &a );
   fail:
    return 0;
  }

Isn't it way nicer that what we have seen before? Additionally, no assignments are performed hence no fuss about goto evilness at all.

Computed goto is a non-standard feature supported by many popular C and C++ compilers. Basically, it allows to store a label into a variable and perform jumps to it. It differs from calling function by pointer because no return is ever performed. The simplest case is shown below:

  #include <stdio.h>

  int main() {
    void* jumpto = &&label;

    goto *jumpto;
    puts("Did not jump");
    return 0;

   label:
    puts("Did jump");
    return 1;
  }

We are taking raw label address using an unusual double ampersand syntax and then perform a goto. Notice the additional asterisk before goto operand. When launched, this program will output Did jump.

Where can we use such a feature? Expressivity wise, it is not very interesting. However, sometimes we can get a speedup. A case that comes to my mind is a bytecode interpreter (but I have written hell of a ton of them, so I should be quite biased towards them). The instruction fetching takes typically no less than 30% of the execution time, and computed goto allows one to speed it up.

Without computed goto:

  #include <stdio.h>
  #include <inttypes.h>

  enum bc { BC_PUSH, BC_PRINT, BC_ADD, BC_HALT };

  uint8_t program[] = { BC_PUSH, 1, BC_PUSH, 41, BC_ADD, BC_PRINT, BC_HALT };

  void interpreter( uint8_t* instr ) {
    uint8_t stack[256];
    uint8_t* sp = stack;

    for (;;)
      switch ( *instr )  {
      case BC_PUSH:
        instr++;
        sp++;
        *sp = *instr;
        instr++;
        break;

      case BC_PRINT:
        printf( "%" PRId8 "\n", sp[0] );
        instr++;
        break;

      case BC_ADD:
        sp--;
        sp[0] += sp[1];
        instr++;
        break;

      case BC_HALT:
        return;
      }
  }

  int main() {
    interpreter( program );
    return 0;
  }

With computed gotos:

#include <stdio.h>
#include <inttypes.h>

enum bc { BC_PUSH, BC_PRINT, BC_ADD, BC_HALT };

uint8_t program[] = { BC_PUSH, 1, BC_PUSH, 41, BC_ADD, BC_PRINT, BC_HALT };

void interpreter( uint8_t* instr ) {
  uint8_t stack[256];
  uint8_t* sp = stack;

  void* labels[] = { &&label_PUSH, &&label_PRINT, &&label_ADD, &&label_HALT };

  goto *labels[*instr];

  label_PUSH:
  instr++;
  sp++;
  *sp = *instr;
  instr++;
  goto *labels[*instr];

  label_PRINT:
  printf( "%" PRId8 "\n", sp[0] );
  instr++;
  goto *labels[*instr];

  label_ADD:
  sp--;
  sp[0] += sp[1];
  instr++;
  goto *labels[*instr];

  label_HALT:
  return;
}

int main() {
  interpreter( program );
  return 0;
}

We replaced switch with an array storing the address of an instruction handler. Each time we fetch an instruction, we are using its bytecode as an offset in this array. After taking an address from there, we jump to it. The larger the instruction set and the switch gets, the more noticeable gets the difference in a real world program.

Using switch slow us down for two reasons:

• It is forced to perform the bounds check according to C standard. If no case exists for a switch expression, and the default case is missing as well, no part of the switch body is executed¹. Computed goto will just result in an undefined behavior in case of an invalid opcode.
• When using switch, there is a single point where the decision about where are we going is taken. In case of computed goto, such decisions are taken at the end of each instruction handler. It makes CPU hold separate prediction histories for each decision making point, which is good for dynamic branch-predicting algorithms.

Starting with Haswell architecture the branch prediction algorithms were tuned so that switch is predicted better, so the performance gain from using computed goto is not that substantial.

P.S. If you really want to make a faster interpreter, consider implementing indirect threaded code, direct threaded code or write a JIT compiler. Computed goto is not a magical thing to make your interpreter as fast as possible.

1. Where to establish the causal relation? We are going to use propositions, which represent quantitative properties of various systems in a given moment of time, that is, measurable properties. Events represents the changes in these properties. These events will be selected as causes and consequences. This way we are building an expressive language to construct statements of a scientific value. Contrary, people often think about events that can either happen or not. In our system such events are easily modeled as Boolean properties (taking value of either 0 or 1).
2. There exist two seemingly equally justified points of view on the place of causal relations in the world.
   • There are common causation laws that rule the world; then they are instantiated in different situations with different events.
   • Causes and consequences are an indispensable part of the world, all generalizations are secondary.

The arguments we are going to study can partly be traced to XVIII century! Meet David Hume, a philosopher.

Hume’s account on causes and consequences is highly empiric. What an observer (even ideal one) can observe about two events A and B is:

• \(A\) occured before \(B\).
• \(A\) and \(B\) are close in space and time. Or they are connected by a chain of events, where each link is a relation between two events which satisfies these properties.
• When we observe something resembling \(A\) again, something resembling \(B\) appears as well.

It is suspicious that these three observations sum up everything an observer can notice — even an ideal observer. There is no way this information might be of a foundation for a strict causal relation as we usually imagine it.

Adding causality changes absolutely nothing here, because it does not give us anything observable.

In fact the pattern above can describe many events not necessarily connected in any way. Hume’s opinion is a great starting point, but now let us also address some problems about it.

Three properties of \((A,B)\) pinpointed above occur not only in cases where we want to establish the relation, but also:

1. When \(A\) and \(B\) have a common cause
2. Just by coincidence
3. By preemption. It means that there is an event \(C\) that occurred before \(A\)

and would have caused \(B\) anyway.

These three major problems are addressed differently depending on how one sees causation in general, which features of the cause-consequence pair are really key. Let’s now speak about those ways.

To this day people tend to think about causation as a deterministic beast. Our knowledge about micro world, however, contradicts it (as many things about micro world contradict common sense). Contrary to some traditional views on causation based on necessity of the cause to bring about its effect, their modern counterparts fit mostly in two categories:

1. Those who base on regular occurrences of \(A\) followed by \(B\).
2. Those who reconstruct the events of real world in the other, purely logical world.

Let’s take a closer look at the second point. In order to construct our logical structures we can use certain sets of rules (they can represent f.e. the laws of nature) of form:

\(\forall x, F \ x \Rightarrow G \ x\)

Here for an event \(a\) the expression  \(F a\) will be substituted by an event instance; \(G a\)  is the consequence. As for the arrow, we can substitute it for either a material conditional (think simple implication) or something stronger like a subjunctive conditional ("If Oswald hadn’t killed Kennedy, someone else would have").

You know, sufficiency is not sufficient for causation. Even if we stick to determinism, the overdetermination (multiple causes) alone is a valid reason to question it. Let us say, \(A\) and \(B\) can both equally cause \(C\) and they occurred simultaneously. What caused \(C\)?

Basing on sufficiency of cause we can deduce:

• If \(A\) was the cause, then \(C\) would not have occurred without \(A\). So \(A\) is not the cause.
• If \(B\) was the cause, then \(C\) would not have occurred without \(B\). So \(B\) is not the cause.

Some people tend to think that it’s rather the cause’s necessity for the consequence that forms a casual relation between them. This way \(A\) caused \(B\) if and only if:

• \(A\) and \(B\) occurred.
• We can assert: "If \(A\) had not occurred, than \(B\) would not have occurred" (we will refer to it as sine qua non, because, well, its  what it is).

I guess it should look like: \(A \land B \land ( \neg A \rightarrow \neg B)\).

It is but a foundation of a longer talk I intend to give in the next post based on arguments of Lewis and maybe Mackie (if I do have time for his book).

  Reasoning about sine qua non is not easy as long as you abandon determinism. Non-determinism, however, goes in pair with probabilities. The general idea of this approach is that causes increase probabilities of consequences in a large variety of contexts:

\(P ( B / AZ ) > P (B / \neg AZ )\)

Here \(Z\) should take into account:

• Common causes of \(A\) and \(B\).
• Preemptive causes of \(B\).

This way preemption problem and common cause problem are addressed, and \(A\) and \(B\) become probabilistically independent.

The hard thing is to choose \(Z\) and a good definition of probability.

For example, take relative frequencies for probability. This way we should exclude from \(Z\) all causal consequences of \(B\) for which \(B\) is necessary. If we do not do it, we just get a wrong inequality \(1 > 1\) (check it!). However, look at it again: to calculate \(P\) we need exactly causal data for which we are building a theory! We have faced a vicious circle that is not easy to break.

As we see, there are loads of interesting subtleties when it comes to a closer study of causation. Numerous attempts were made to exile this relation completely from scientific thinking, but its roots are so strong it proved almost impossible to do. Next time I want to talk about Lewis’s very influential theory on causation, dated 1973, and then about his new causation theory of early 2000’s.

The idea is simple: if types are equal, there exists a bijection between their elements. Let us prove a simple lemma exists_bijection:

\[\forall A \ B: Set, A = B \rightarrow \exists f \exists g: \forall x:A \ \forall y : B, (f \ x = y -> g \ y = x )\]

Definition id {T} := fun x : T => x.
 
Lemma exists_bijection (A B: Set):
A = B -> exists f, exists g, forall (x:A) (y:B),
 f x = y -> g y = x.
Proof.
 intro H; subst. 
 exists id. exists id.
 intros. subst. 
 reflexivity.
Qed.

The proof was a piece of cake. Now to battle!

You remember, that unit is a type inhabited by only one element tt?

Inductive unit : Set := tt : unit.

Both f true and f false are evaluated to tt. There is not much choice here. However we know, that \(\forall x, (g \cdot f) \ x = x\) . By instantiating x with true and false we get \(g (f \ true) = true\) and \(g (f \ false) = false\). Since both \(f \ true\) and \(f\ false\) are equal, we deduce \(g\ tt =true\) and \(g \ tt = false\), contradiction.

Now the code:

Lemma unit_neq_bool: bool = unit -> False.
Proof.
intro Heq.
destruct (exists_bijection _ _ Heq) as [f [g Hfg]].
destruct (f true) eqn: Hft.
destruct (f false) eqn: Hff.
pose proof (Hfg _ _ Hft) as Hgtt.
pose proof ( Hfg _ _ Hff) as Hgtf.
 
rewrite Hgtf in Hgtt.
inversion Hgtt.
Qed.

So, the key idea is to enumerate possible bijections. As at least one of sets is finite, you will eventually enumerate all the candidates, and when the sets are of different size, you will get contradictions because you will run out of distinct functions trying to enumerate all bijections.