Multics 2nd Generation (Multics^2):

    Lehman and Belady have studied the history of successive releases in
    a large operating system.  They find that the total number of
    modules increases linearly with release number, but that the number
    of modules affected increases exponentially with release number.
    All repairs tend to destroy the structure, to increase the entropy
    and disorder of the system.  Less and less effort is spent on fixing
    original design flaws; more and more is spent on fixing flaws
    introduced by earlier fixes.  As time passes, the system becomes
    less and less well-ordered. Sooner or later the fixing ceases to
    gain any ground.  Each forward step is matched by a backward one.
    Although in principle usable forever, the system has worn out as a
    base for progress.  Furthermore, machines change, configurations
    change, and user requirements change, so the system is not in fact
    usable forever.  A brand-new, from-the-ground-up redesign is
    necessary.

	-- Fred Brooks, "The Mythical Man-Month", about IBM OS/360
	(but it may as well be applied to Unix too, and perhaps even the
	original Multics)

We Unix people have been deceived, and are deceiving ourselves in the
worst way possible for all too long a time.  One of our core
philosophies is backwards / inside out / upside down / flat-out wrong:

	In unix it's not really true that "everything is a file".

	Rather it is that "everything is an I/O channel, even including
	files"

We may as well still be computing with all our data accessed via
magnetic tape!

In Multics the filesystem is merely a logical map of all memory segments
stored in the system, each of which may be mapped into a process'
address space (assuming appropriate access rights).  Meanwhile real I/O
in Multics is done via a unified, universal, streams-oriented, and
hardware-independent, interface that even works transparently with files
as the "device".  Multics even allows easy interchange of devices while
programs are executing.

In Unix everything is a file descriptor (but a file descriptor is not
necessarily a file, especially not for sockets and pipes),
i.e. everything (except memory) in Unix is I/O, even files.  Files must
be accessed as if they are I/O streams (though they are seek-able, so
somewhat randomly accessible, but that the extreme cost of extra system
calls and far more complex algorithms).  Unix file I/O is no better than
ancient magnetic tape storage, at least w.r.t. algorithm design.  (I'm
ignoring mmap() because it is far too hobbled to be truly useful as a
proper replacement for file access via I/O streams.)

Of course in Multics everything is also a file(name), effectively, since
files are just segments (the I/O subsystem plays tricks with names that
could easily be bound into the same namespaces as data files, though at
the time that didn't seem so important in the Multics world).  However
in Multics the data in a file is just more address space:  data is
directly addressable and thus randomly accessible, and it appears on
demand in the process address space without any thought as to where it
was stored.  We only think in terms of linear streams of data in Multics
when we are forced to communicate with the world outside the process
execution space.  But that's OK because it doesn't limit what the
underlying communications hardware can do, and it fits exactly with the
paradigms implied by the underlying hardware and how it links to the
outside world.  Trying to think of a serial port as an array of data can
get you into trouble and it leaves out some extremely critical concepts
about the very nature of communications.  (Hmmmm.... this probably means
Multics with DMA hardware could be made more secure too!)

Paging vs. Segments

The purpose of paging is to make the allocation of physical memory
easier.  One may think of paging as the intermediate ground between a
fully associative memory, having each word addressed by means of some
part of its contents, and a normal memory, having each memory location
addressed by a specific integer forever fixed to that physical location.
In paging, blocks of memory are assigned differing base addresses.
Addressing within a block is relative to the beginning of the block.
Thus if association and relative addressing are handled with a break
occurring within a normal break of the word (viz. in a binary machine
block size is a power of 2), then a number of noncontiguous blocks of
memory can be made to look contiguous through proper association.  The
association between a block and a specific base address can be
dynamically changed by program during the execution of appropriate parts
of the executive routine.

Segments are used not for the allocation of physical memory, but for the
allocation of address space.  A segment defines some object such as a
data area, a procedure (program) or the like.  In a sense, each segment
corresponds to a part of virtual memory whose size is whatever size, up
to a maximum limit, that is required for its content and which can
expand or contract as that size changes.  In theory, as many such
segments can be available to a programmer as necessary.  Observe that
although segments and pages are two distinctly different entities, they
work together to facilitate the allocation of physical memory and
virtual memory.  Although a large number of segments may be defined,
each one having a large number of words, only the currently referenced
pages of pertinent segments need to be loaded in physical memory at any
time.

(The above section is paraphrased from "System Design of a Computer for
Time Sharing Applications" by E. L. Glaser, J. F. Couleur, and
G. A. Oliver.  This paper gives a good abstract overview of the rational
and design of the "extension" features required to support Multics on
the GE-635 -- i.e. the creation of the GE-645.)


Multics securty:  The 1970's is the 2020's

The high-assurance, military security, mainframe oriented model of the
Orange Book is oriented toward the problems of the 1970's.  One can
assume that Multics systems ran on trusted hardware that was maintained
and administered by trained and trusted people.  Users didn't buy
graphics cards on Canal Street and plug them into the mainframe.  In
many ways, the security design of Multics-based systems pushed a lot of
hard-to-solve issues off onto Operations and Field Engineering.  Multics
hardware originated from a trusted manufacturer and was shipped on
trusted trucks, installed by trusted FEs, and kept in a locked room.

(above paraphrased from http://www.multicians.org/b2.html, "4.5
Relevance to current and future security needs")

Well what do you know!  Those same needs and assumptions can be applied
to modern Internet SaaS platforms.

Multics and its security as a precursor and model for "cloud computing"

	In terms of past examples, the substantial Multics efforts over
	a number of years by Honeywell (initially General Electric) is
	one of the few.  Decades in advance of widespread commercial
	need Multics addressed the challenge of creating a "computing
	utility" for which security was a central value proposition.
	Many decades later essentially this vision has been given the
	new name of "cloud computing" – unfortunately without
	significant attention to security.

	https://intelligence.org/2014/06/23/roger-schell/



Original Multics Goals:

- Avoid multiple copies of the same information in memory

- Avoid copying or processing each word of memory repeatedly

- Control sharing at the segment level

- Link subroutines incrementally on demand

- Support symmetric shared-memory multiprocessing

- Support programming in multiple languages


Multics^2 should have:


* Single Level Store with a hierarchical filesystem.

  - Einstein inspired Peter G. Neumann to think in terms of hierarchical
    systems, which in turn begat the Multics filesystem, and thus Unix
    and all subsequent similar hierarchical filesystems.

  - block/page level de-duplication with copy-on-write semantics for all
    file copies

  - memory segments are files and vice-versa -- one-to-one with only a
    hackish streams-based method for accessing segments as an I/O stream
    for external "serial" communications (and perhaps the benefit of
    silly old POSIX-compatibility).

  - All information stored in memory must be directly addressable by any
    process.  I.e. everything that is "just" data, should be accessed
    directly via pointers into memory.  Only interfaces where data must
    be serialized for some reason, e.g. network communications, ttys,
    etc. should be done via some I/O stream mechanism.

  - Access control can be performed at each reference to memory, or at
    least at each mapping of a page of memory.

  - abstractions such as buffering in STDIO are pure overhead!


* Extensions to C that work similar to the old extensions to PL/1 for
  multics?  Or just stick to pointers?  We already depend almost
  entirely on malloc() and mmap() which return bounded regions, and we
  even expect that if we have to grow them that they may move.

* also possibly offer a tagged, search-based filesystem overlay

  - http://eecs.harvard.edu/margo/papers/hotos09/paper.pdf


* natural dynamic linking with a search path handled by the link(call)
  trap handler, and pure shareable compiled code (i.e. no modification
  of code segments is allowed or required).

* ability for ALL programs to make all non-basic dynamically linked
  libraries and "frameworks" entirely optional!  The most basic
  functionality MUST NOT require installing all possible bloatware that
  a program might want to use to implement all of its options.


* designed for real-world general purpose use, as well as for research

  - fully POSIX-compatible runtime emulation with fread() and fwrite()
    being effectively memcpy().


* in general it should try to fix or avoid all the "stupid" things in
  various systems APIs


In general the purpose is not to be a research OS, but rather to be a
re-implementation of ideas that have fallen by the wayside.

BTW, Chris offers a good discussion of what "success" means for a
research OS:

	https://utcc.utoronto.ca/~cks/space/blog/tech/SuccessForResearchOSes


Ideas:


In many ways Multics^2 is very similar to The Amber Operating System:

	http://www.mit.edu/~cbf/thesis.htm

Transform Xen into, or otherwise use a hypervisor to implement, Multics^2

    - idea from MirageOS (http://queue.acm.org/detail.cfm?id=2566628)

    - maybe each process is a whole virtual machine

    - you run either a small runtime, or a whole user-dedicated OS, in
      each VM

    - threads are always virtual-CPUs in the VM?

    - use unix-like job control to have multiple threads running in one
      process?

    - the hypervisor implements an emulation of segments for a slightly
      enhanced virtual-CPU+MMU even if the underlying CPU doesn't do
      segments

    - this is, perhaps unfortunately, much more in keeping with the
      heavy-weight nature of the original Multics processes, though if
      multiple vCPUs can be assigned to each domain then threading is
      available and good inter-thread communications, e.g. like
      goroutines and channels, could ease the pressure on having
      multiple processes for a given task.

    - See also EthOS:

	http://www.ethos-os.org/
	https://www.cs.uic.edu/~spopuri/ethos.html

Or maybe just use hypervisor technology to implement rings?

On the other hand Jonathan Shapiro said:

	"The idea of adapting paravirtualization to a microkernel for
	use within an operating system is, frankly, silly. Several
	existing microkernels, including L4.sec (though not L3 or L4),
	KeyKOS, EROS, Coyotos, and CapROS, have isolation and
	communication mechanisms that are already better suited to this
	task."

	http://www.coyotos.org/docs/misc/linus-rebuttal.html

- but then Shapiro isn't known for being open-minded about some OS
  concepts.  :-)


Rings have a huge advantage when there's hardware ring protection:

	Multics can do a direct procedure call to a higher privilege
	level, with the full generality of high-level language argument
	passing, without taking a trap or invoking mediation code.  The
	return from higher privilege (lower ring) to lower privilege
	(higher ring) is also direct.

  - the kernel code is directly shared with all processes, i.e. kernel
    code segments are mapped into the address space of each process --
    just another segment, or several.

  - A process can call kernel functions directly using normal function
    call mechanisms -- there are no "system calls" as in the unix sense.


For a practical experiment a 64-bit address space should suffice, even
if rings are in software (though we would need multi-segment files):

	 2 bits ring #
	30 bits segment #
	32 bits segment address

(are there any modern large-scale data apps that expect more than
32-bits of address space for individual arrays?  yes, but not many)

If we want to avoid multi-segment files and we can do rings in hardware:

	24 bits segment #	(16 million segments per process)
	40 bits segment address (1,099,511,627,776, 1 terabyte)

or even:

	 4 bits ring #
	20 bits segment #	(1 million segments per process)
	40 bits segment address (1,099,511,627,776, 1 terabyte)

or:

	 2 bits ring #
	22 bits segment #	(4 million segments per process)
	40 bits segment address (1,099,511,627,776, 1 terabyte)

We can also split the address space into regions using "flag" bits in
pointers, and thus also avoid needing multi-segment files at the same
time!:

	Highest bit	63:	0 = regular address, 1 = memory mapped files
				(i.e. is this a "segment" address?)

	next highest	62:	very large files (or just if any one of
				the VLF# bits is non-zero -- requires
				masking and testing for non-zero as
				opposed to testing just one bit)

	next 8 bits	54-61:	VLF file number (256 very large files open)
      				2^52 byte max (4,503,599,627,370,496) 4 petabyte

	next 21 bits	33-53	regular file numbers (1,048,576 files)
				2^32 byte max (4,294,967,296) 4 gigabyte

          1         2         3         4         5         6
0123456789 123456789 123456789 123456789 123456789 123456789 123
                                 R                    V       VF
                                   E                    L     LI
                                     G                    F   FL
                                       #                    # ?E
                                                               ?

Mask off the flags and file number and you have the index (offset).

Take 2 bits from the regular file numbers to do rings in software.

Get rid of ordinary file descriptors, have open(2) return a pointer ala
mmap(), and have socket(2) be the only call that returns a file
descriptor.

So, how do we do dynamic linking with this scheme?



Notes about 80386:

I've been doing an awful lot of reading and research about Multics again
recently, including looking at the x86 segmentation and protection model
in view of how it could work for Multics.

So far as I can tell IA-32 is more or less _exactly_ what is needed for
Multics, or a Multics-like system, with some possible limitations.

    A virtual address consists of a 16-bit (13+1 are address bits)
    segment selector and a 32-bit segment offset.  The selector is used
    to fetch a segment descriptor from a table (actually, there are two
    tables and so the "plus one" of the bits of the selector is used to
    choose which table) (and the other two bits are the Requested
    Privilege Level).  The 64-bit descriptor itself contains the 32-bit
    address of the segment (called the segment base) 21 bits indicating
    its length, and miscellaneous bits indicating protections and other
    options.  The segment length is indicated by a 20-bit limit and one
    bit to indicate whether the limit should be interpreted as bytes or
    pages.  (The segment base and limit "fields" are actually scattered
    around the descriptor to provide compatibility with earlier version
    of the hardware.)  If the offset from the original virtual address
    does not exceed the segment length, it is added to the base to get a
    "physical" address called the linear address.  If paging is turned
    off, the linear address really is the physical address. Otherwise,
    it is translated by a two-level page table as described previously,
    with the 32-bit address divided into two 10-bit page numbers and a
    12 bit offset (a page is 4KB).

Warning:  Apparently the Atom processors have very sucky performance,
especially when using non-zero-based code segments.

The main addressing limitation in 32-bit x86 is the relatively small
13-bit segment index held in the segment registers.  With the two
possible descriptor tables that gives an absolute maximum of 2^14-1
segments per process, though practically I'd guess the GDT is only
useful for code segments and possibly a very few truly globally needed
file segments, so that leaves just 8192 segments for stack, data, and
general file access by each process.  That's still twice what the
practical limit was for packed pointers on the old Honeywell 6180, and
4-8 times the average number of open file descriptors permitted in the
average modern unix-like system, so perhaps it's not really an important
limit.  Perhaps there's some convenient way for a process to switch
LDT's on demand too.  Also, it's between 2 and 8 times the number of
open file descriptors generally limited to by modern Unix systems.

Obviously the x86 only has 4 rings, but I don't currently see that as a
problem.

I think all x86-64 implementations still have full compatibility modes
for x86 (IA-32, i.e. 32-bit) protected mode addressing with full
segmented addressing and paged segments.  They pretty much have to in
order to run all 32-bit software.  (FYI, the FS and GS registers retain
their use as segment descriptor table indexes even in 64-bit mode
(i.e. they can still have non-zero base addresses), but without any real
protection features, I think.)

Paul Green's Multics VM paper suggests another, possibly more efficient,
way to avoid multi-segment files:

     Concatenating segments would be simple if incrementing the word
     offset past the last word carried into the segment number.  You
     wouldn't want this by default, but with a bit in the SDW you could
     control which segments were concatenated and which were not.

So, can we modify x86 segmentation to allow concatenated segments?  This
would allow direct, trivial, access to files larger than 4GB without
having to re-invent multi-segment files again.  There should still be a
bit available in a segment descriptor to indicate that it is to be
concatenated with the following descriptor.  Getting pointer arithmetic
to work right might be tricky though.

  - "segment" registers point into a segment descriptor table (SDT):

          2 bits ring # (requestor's privilege level)
	  1 bit segment descriptor table indicator (global vs. local)
	 13 bits segment descriptor index (8192 segments per SDT)

	 32 bits segment address (offset)

  - segment descriptors (SDT entries, or SDWs in Multics parlance):

	 16 bit segment limit (in either bytes or 4k pages, 4GB max)

  - can a process reload its own LSDT without a ring#0 call?


Ideally though the x86_64 would have segmentation re-enabled, thus
doubling the size of index registers and hopefully also doubling the
size of segment selector registers (though maybe not increasing the
descriptor table size by the same power of two -- other flag bits might
be useful in the visible part of segment selector registers), and of
course adding at least 4 more bytes (but ideally doubling) the size of
segment descriptors.


IA-32 TSS:

- dynacube OS uses just two TSS segments in the GDT:

  http://dynacube.net/


Note about Honeywell 6180 limits:

  - in practice packed pointers were used with a limit of 4096 segments
    per process.


Other CPUs:

CHERI

  Memory Segmentation to Support Secure Applications

  "A segment mechanism that implements the capability model of safe,
  programmatic memory protection"

  http://www.cl.cam.ac.uk/research/security/ctsrd/pdfs/2013essos-cheri.pdf

  "CHERI can do anything Multics could do: segmentation, paging,
  dynamic linking, ring-structured software"

   -- Peter G. Neumann in "An Interview with..." by Rick Farrow in
   ;login:, Winter 2017 vol. 42, no. 4


More about "Capabilities"

	Capability-Based Computer Systems Henry M. Levy This book was
	published by Digital Press in 1984.  It is still the most
	thorough survey and description of early capability-based and
	object-based hardware and software systems.  The book is now out
	of print and the copyright belongs to the author, who makes the
	material available here for viewing or downloading, in Adobe
	Acrobat PDF format:

	https://homes.cs.washington.edu/~levy/capabook/

- see also Plessey System 250

- capabilities vs. access control lists:

	J. H. Saltzer and M. D. Schroeder. 1975. The protection of
	information in computer systems. Proceedings of the IEEE 63, 9 (
	Sept. 1975), 1278–1308.

  They thought revocation of access was one of the major area where
  capabilities fell short.

  In segmentation and with the introduction of "generation"
  numbers in the segment table revocation is simply a matter of
  incrementing the generation number.

  See also

	"Using segmentation to build a capability-based single address
	space operating system", Wuyang Chung <wy-chung@outlook.com>

	https://www.bsdcan.org/events/bsdcan_2023/sessions/session/129/slides/50/slides.pdf

Mill

	https://millcomputing.com/


I'm also extremely interested in exploring what hypervisor support in
modern x86 CPUs might bring to the table in terms of building a new
Multics-like system.  What if each process is a virtual machine or maybe
even an LPAR (but with overlapping memory, of course)?  Virtual CPUs as
threads?

I think the PowerPC might work too (with software rings?), and I'm also
interested in seeing how difficult it might be to add segmentation and
ring "hardware" to an ARM, SPARC, or MIPS architecture, perhaps as an
experiment in an FPGA implementation.  There's also a patent I found
about emulating segment bounds checking in a simpler RISC-style MMU:
https://www.google.com/patents/US5652872



For now IS IT possible to just do segment + offset calculations manually
on non-segmented architectures?

    - how might CPU support features for a hypervisor help?

    - does an MMU help emulate segments?

    - segments as pages?

How did the S-1 support a variable boundary between the segment number
and the offset?  (S-1 segments could take up the whole address space of
the system, apparently.)


Here's a linux extension that provided multiple global address space
mappings for files via mmap():

	Srinivas Palnati, Gautam Barua, "Persistent Storage for 64-bit
	Systems", International Conference on Information Technology
	(CIT-2001), Gopalpur-on-the-sea, India, December 20-22, 2001.

Is there any performance advantage to loading a whole library in one
segment (when there are no adverse security implications?).  Otherwise
every independent function gets its own segment (we would still have a
billion to play with on a 64-bit address even with 2-bits reserved for
rings!).

    - probably yes -- at least for when the library calls lots of other
      members of itself, since then internal calls won't need fixups on
      the initial invocation.

MMU Paging systems can be used to implement segments.  From the Amber
paper[1]:

	Architectures that have the necessary hardware for demand paging
	and a large address space (VAX and IBM 370 with Extended
	Addressing option) can still use the techniques of segmentation
	by allocating contiguous ranges of pages to represent a segment.
	They will not however, have the feature of trapping references
	that overflow a segments boundaries that the Multics and S-1
	hardware provide.  This feature is primarily a debugging aid,
	but it can be an important one.

however there are problems with only having page tables:

	On a segmented machine like Multics, the S-1, or the Prime 500
	series, access to a segment is determined from bits in a user's
	segment descriptor word.  On a paged system each page table word
	has access bits.  (The access we are referring to here is read,
	execute, or write to an area of main memory. The S-1 actually
	has bits in both the segment and the page table words and it
	and's them together to determine effective access.)  This
	unfortunately means that two users who have the same segment
	mapped in with different access rights cannot share the same
	page table.


Files vs. I/O -- everything is a file (descriptor) vs. SingleLevelStore

	File systems have (at least) two undesirable characteristics:
	both the addressing model and the consistency semantics differ
	from those of memory, leading to a change in programming model
	at the storage boundary.  Main memory is a single flat space of
	pages with a simple durability (persistence) model: all or
	nothing. File content durability is a complex function of
	implementation, caching, and timing.  Memory is globally
	consistent.  File systems offer no global consistency model.
	Following a crash recovery, individual files may be lost or
	damaged, or may be collectively inconsistent even though they
	are individually sound.

	from the abstract of:
	"Design Evolution of the EROS Single-Level Store"
	by: Jonathan Shapiro, Jonathan Adams


Issues with using memory as storage, at least with respect to MMAP:

- note that with a Single Level Store all views of the data are
  implicitly guaranteed to be unified.

- a system all similar to msync(2) could be used to "secure" (flush) all
  writes to a segment, thus careful use of it (like a mutex of sorts),
  combined with some effect of something akin to mlockall(2) to
  guarantee that no writes happen until the msync() call, could
  guarantee write order to secondary storage, though of course efficiecy
  here requires that writing of a whole page is as efficient as writing
  some small part of the page (normally the case if I/O is done with DMA
  transfers I think)  [[ how could this be made more transparent? --
  perhaps by providing a write(2) like interface? ]]

- it is hard to escape from point #3, though presumably that's algorithm
  dependent.

- similarly point #2, but that is in part a language problem.

	From: Howard Chu via Cyrus-devel <cyrus-devel@lists.andrew.cmu.edu>
	To: Cyrus Devel <cyrus-devel@lists.andrew.cmu.edu>
	Message-ID: <1c46d2af-c16f-aeb3-9e32-291d3ac4e216@highlandsun.com>
	Date: Fri, 30 Nov 2018 11:09:23 +0000

	I've covered the reasons for/against writing thru mmap in my
	LMDB design papers.  I don't know how relevant all of these are
	for your use case:

	1. writing thru mmap loses any control over write ordering - the
	   OS will page dirty pages out in arbitrary order.

	   If you're using a filesystem that supports ordered writes, it
	   will preserve the ordering of data from write() calls.

	2. making the mmap writable opens the possibility of
	   undetectable data structure corruption if any other code is
	   doing stray writes through arbitrary pointers.  You need to
	   be very sure your code is bug-free.

	3. if your DB is larger than RAM, writing thru mmap is slower
	   than using write() syscalls.  Whenever you access a page for
	   the first time, the OS will page it in.  This is a wasted I/O
	   if all you're doing is overwriting the page with new data.

	4. you can't use mmap exclusively, if you need to grow the
	   output file.  You can only write thru the mapping to pages
	   that already exist.  If you need to grow the file, you must
	   preallocate the space, otherwise you get a SEGV when
	   referencing unallocated pages.

	And a side note, multiple studies have shown that skiplists are
	not cache-friendly, and thus have inferior performance to B+tree
	organizations.  A skiplist is a very poor choice for a read/write
	data structure.

	Obviously I would recommend you use something carefully designed
	and heavily tested, like LMDB, instead of whatever you're using.

	There's one point in favor of writing thru mmap - if you take
	care of all the other potential gotchas, it will work on every
	OS that implements mmap.  Using mmap for reads, and syscalls for
	writes, is only valid on OSs with a unified buffer cache. While
	this isn't a problem on most modern OSs, OpenBSD is a notable
	example of an OS that lacks this, and so that approach always
	results in file corruption there.

	-- 
	  -- Howard Chu
	  CTO, Symas Corp.           http://www.symas.com
	  Director, Highland Sun     http://highlandsun.com/hyc/
	  Chief Architect, OpenLDAP  http://www.openldap.org/project/


- Bron replied to say:

	This is not a concern at all - twoskip is deliberately designed
	such that it does a single write and then flush to "dirty" the
	file, all changes made while dirty are fully revertable if it
	crashes, and then it does a fsync (msync now I guess!) before a
	single write which clears the dirty flag. So long as a single
	256 byte write is consistent, it's safe.



hardware transactional memory

	 https://dl.acm.org/citation.cfm?id=2618405
	 https://fsgeek.ca/2018/04/27/the-future-of-synchronization-on-multicores-the-mulitcore-transformation/

Security

What are the modern threats to a Multics-like system that are different
from what it faced originally?  How should it deal with bugs in programs
responsible for the interpretation of structured content such as
graphics (HTML, etc., etc., etc.)

- security sensitive processes could be prevented from mapping in new
  code segments just before they start running (and of course it is an
  error to try to jump to a data (or stack) segment, and the stack(s)
  and heap segment(s) do not share the same address(offset) space).

- separate processes for separate functional purposes, e.g. browser in
  one, mail in another, DBs each in their own, compiler is run in its
  own, etc.  Each network server in its own, etc.  Even an editor like
  Emacs could run in its own process.


How to deal with the administration issue -- i.e. how much sysadmin can
be automated to the degree that a non-expert can own and run a machine?


Jonathan S. Shapiro argues against segments in:

	 http://seclists.org/risks/2014/q2/1

How distributed can it all be?

- e.g.:  like Apollo Domain/OS and/or Amoeba


- full virtual cloud for all computing devices?

  - any computer can join the cloud, local resources can be shared or
    private, some other resources (filesystem and CPU, etc.) can be
    "more trusted" than the whole cloud.

  - use something like Apple's security processor to make one device the
    only thing that can decrypt some segments

    - additionally be able to store keys into it to allow identities to
      share devices, migrate between devices, make devices "disposable"



What to write it all in?

- some "safe" C?  with modules

  - see ~notes/C


- why not plain C?  "Undefined behaviour".  It is a thing "defined" for
  C, for one:

    ... we have C compiler maintainers who assert that any behavior not
    defined in the C standard is not something where they have to pick
    one behavior and stick with it, but can instead declare it an
    impossible situation, that code which relies upon common assumptions
    is [therefore] buggy, and proceed to optimize away the security checks.

    If I program in C, I need to defend against the compiler maintainers.
    If I program in Go, the language maintainers defend me from my mistakes.

  Phil Pennock
  https://bridge.grumpy-troll.org/2017/04/golang-ssh-radix/

  A better language would not define anything as "undefined behaviour", ever!

  So, not Standard C as defined by modern standards.


- Hare, https://harelang.org/

  - "Hare definitely does not have is the same kind of 'undefined
    behavior' that C compilers use as a license to do whatever they want
    to your program"

  - "all strings are always valid UTF-8.  We also store the length
    alongside the string, allowing NUL to appear in the string contents,
    and for len(str) to be an O(1) operation."

  - can be useful everywhere C is useful, BUT...

    - it lacks multithreading in the standard library


- Go, with extensions and a different runtime?

  - can a Go-like language automatically and reliably detect when, e.g.,
    features are used which might depend on a hosted runtime (as opposed
    to the bare metal runtime), e.g. garbage collection.

  - see GO-OSe and bootgo

	https://github.com/boomshroom/goose
	https://github.com/jjyr/bootgo


- V  https://vlang.io/  https://github.com/vlang/v

  - hmmm.... this V language is very simple, much like Go, but without
    (currently) a garbage collector -- cleanup is done dynamically when
    functions return, etc.

	V is a statically typed compiled programming language designed
	for building maintainable software.

	It's similar to Go and is also influenced by Oberon, Rust, Swift.

  - Has enums!  (and even generics!)

  - super-safe:

  	- No undefined values
	- No undefined behavior
	- Bounds checking

	There's no garbage collection or reference counting.  V cleans
	everything up during compilation.  If your V program compiles,
	it's guaranteed that it's going to be leak free.

  - there's a C to V translator and C interoperability is easy

  - claims compiled binaries have no dependencies (but don't seem to be
    static-linked -- they're too small, for one)

  - currently implements concurrency with system threads, but claims to
    be adding a coroutines and a scheduler

  - "V"ery interesting!  Extremely good candidate!

  - possible drawbacks:  difficult to manage pointers?


- there's also Jai:  imperative static/strongly typed C-style language
  by Jonathan Loom

	https://inductive.no/jai/
	https://github.com/BSVino/JaiPrimer/blob/master/JaiPrimer.md

  No implicit type conversions
  No header files
  Jai code can be built on top of existing C libraries

  - need to know more about its runtime


- Nim

	https://nim-lang.org/

  - it's a bit/very annoyingly python-like in look

  - it has no runtime compiler library -- executables are stand-alone!

    * Nim generates native dependency-free executables, not dependent on
      a virtual machine, which are small and allow easy redistribution.

    * Fast deferred reference counting memory management that supports
      real-time systems.

    * Nim is self-contained:  the compiler and the standard library are
      implemented in Nim.


- Zig -- http://ziglang.org/ (by Andrew Kelley)

  ready and running, apparently.  BUT oh, gawd -- it's currently written
  in C++ and requires CMake to build the compiler (because it is a thin
  front-end to LLVM) (though apparently comes with its own built-in
  build system so that Zig programs don't need anything like CMake,
  etc.)

  BUT, this may be changing (divorce from LLVM):

	https://kristoff.it/blog/zig-new-relationship-llvm/

  See also:

	https://lists.sr.ht/~sircmpwn/public-inbox/%3C874kh7ovmk.fsf%40dismail.de%3E

  Some people think Zig is too far away from being C-like to be "good".


- Eiffel:  https://www.eiffel.org/  too wordy, and possibly suffering
  the same kind of runtime issues as Go.


- Rust: http://rust-lang.org/ (overly excessively complex and baroque,
  so, probably not)


- D: http://dlang.org/ (overly complex, probably not)


- Ocaml (probably not -- too different)


- XC -- Xmos C

  - C with Communicating Sequential Processes on multi-core hardware

  - was heavily influenced by the occam programming language

  https://en.wikipedia.org/wiki/XC_(programming_language)


- something with unsigned integers by default, which can be specified in
  exact bit widths (a bit like PL/1)


- What about Turing Plus?  Still available and maintained:

  - http://research.cs.queensu.ca/~cordy/pub/downloads/tplus/

  - only for i386(?)

  - somewhat wordy (pascal-like begin/end, etc.)

  - but still maybe better than Oberon/Modula-2/whatever?


- Scheme?

  - there are several good Scheme-to-C compilers:

	https://justinethier.github.io/cyclone/
	http://gambitscheme.org/wiki/index.php/Main_Page

  - what about a multics-like OS with a garbage collector in the
    "kernel"???  Why not?


- Ada?

  e.g. as was done for: https://wookey-project.github.io/ewok/index.html

  - investigate Ada's development:

  "DEPARTMENT OF DEFENSE REQUIREMENTS FOR HIGH ORDER COMPUTER
   PROGRAMMING LANGUAGES"

  * STRAWMAN issued in April 1975
  * WOODENMAN issued in August 1975
  * TINMAN issued in January 1976
  * IRONMAN issued in January 1977 (revised in July 1977)
  * SANDMAN not published but circulated in January 1978
  * STEELMAN issued in June 1978
  * PEBBLEMAN issued in July 1978
  * PEBBLEMAN Revised and issued in January 1979
  * STONEMAN issued in February 1980


What to run it on:


Standards:

- which to follow, which to ignore, which to violate on purpose?

- UTF-8?  UTF-32LE? (or UTF-32BE), but UTF-32* still does not have a
  fixed byte count per displayed character due to combining characters.
  (also up to four times the size of UTF-8 depending on how many of the
  characters are in the ASCII subset) On Unix systems, UTF-32 strings
  are sometimes used for storage, due to the type wchar_t being defined
  as 32 bit.

Here's an old paper by the unix group defending the LP64 programming for
64-bit architectures, and to some extent espousing the very large
address space of 64-bit pointers:

	http://www.unix.org/version2/whatsnew/lp64_wp.html

HP claims to be designing a new architecture computer with a "flat"
memory design much like Multics needs:

	https://www.hpl.hp.com/research/systems-research/themachine/


People:

- Multicians

- Karl Aurbach

- Wayne Radinsky


See Also:


Plessey System 250

- https://en.wikipedia.org/wiki/Plessey_System_250


SOMBRERO

AMOEBA:  Tanenbaum et al

- uses purely the concepts of "objects" and "capabilities", with any
  style of filesystem (implemented by a set of three user processes
  running on each CPU) mapped on top of these.

- AMOEBA was independently improved by Stefan Bosse to create FSD-AMOEBA

	http://fsd-amoeba.sourceforge.net/

  which then begat:

	VAM (Virtual Amoeba Machine) - A virtual machine for distributed
	systems based on OCaML and the Amoeba OS (2000-2006)

	Vertex Amoeba - An extended and updated version of the
	distributed Operating System Amoeba from the Vrije University
	for x86 platforms and embedded PCs (1999-2004)


MUNGI: A Distributed Single Address-Space Operating System

- "private per-process data can therefore only be provided by using
  relative addressing from a base register."

  So why not just use segments?  :-)

- papers provide good consideration of many issues w.r.t. distributed
  operation

- doc archive at http://www.nic.funet.fi/pub/doc/OS/Mungi/

  No. 9302
  issued in March, 1993
  Title:
  A Distributed Single Address-Space Operating System Supporting Persistence

  Author(s):
  Gernot Heiser, Kevin Elphinstone, Stephen Russell, Graham R. Hellestrand

  Abstract:
  Persistence has long been difficult to integrate into operating systems.
  The main problem is that pointers lose their meaning once they are taken
  out of their address-space. We present a distributed system which has a
  single address-space encompassing all virtual memory of every node in
  the system. This design has become possible (and practicable) with the
  advent of 64-bit microprocessors.

  In our system, every pointer retains its meaning independent of its
  location, even across nodes or on secondary storage. No restrictions are
  imposed on the use of pointers by application programs. Hence
  persistence is naturally and elegantly integrated into the system.
  Further features are uniform addressing and unlimited sharing of data,
  and memory protection based on password capabilities, making the system
  easy to use. A reliable paging protocol ensures that the impact of node
  crashes on other parts of the system is minimised.

  No. 9314
  issued in November, 1993
  Title:
  Mungi: A Distributed Single Address-Space Operating System
  (The text of this report has been acepted for ACSC-17)

  Author(s):
  Gernot Heiser, Kevin Elphinstone, Stephen Russell, Jerry Vochteloo

  Abstract:
  With the development of 64-bit microprocessors, it is now possible to
  combine local, secondary and remote storage into a large single
  address-space. This results in a uniform method for naming and accessing
  objects regardless of their location, removes the distinction between
  persistent and transient data, and simplifies the migration of data and
  processes.

  This paper describes the Mungi single address-space operating
  system.  Mungi provides a distributed single level address-space, which
  is protected using password capabilities. The protection system performs
  efficiently on conventional architectures, and is simple enough that
  most programs do not need to be aware of its operation.

MONADS

 A capability-based distributed shared memory.
 F. A. Henskens, J. Rosenberg, and J. L. Keedy.
 In Proceedings of the 14th Australian Computer Science Conference, pages 29.1–12, 1991.

- MONADS depends on specialised hardware


LOCUS

  LOCUS: a network transparent, high reliability distributed system
  G. Popek, B. Walker, J. Chow, D. Edwards, C. Kline, G. Rudisin, and G. Thiel.
  In Proceedings of the 8th ACM Symposium on OS Principles, pages 169–77, 1981


OPAL

http://homes.cs.washington.edu/~levy/opal/opal.html

- supports a huge (64-bit) global (network-wide) "pure" address space
  divided into virtual segments which are named by capabilities.  Shared
  code is in segments called modules.  All segments are potentially
  persistent and recoverable.  Persistent segments create a virtual
  filesystem as a single level store.

- the paper notes disadvantages of hardware-based segmented addressing
  in the likes of multics:  (1) cross-segment pointers are impossible;
  (2) multiple pointer forms must be treated differently by application
  programs; (3) software must be used to also co-ordinate the use of
  segment registers to create the illusion of a single address space.
  (but that last point is silly because in a global pure address space
  software must be used to co-ordinate the paging table and VM hardware)

- a major disadvantage of virtual segments is that copy-on-write is
  impossible (for data segments containing internal pointers) without
  supporting some form of "pointer conversion" (because internal
  pointers are always global pointers, never offsets from the segment
  base)

- note that in Multics cross-segment pointers are actually possible if
  there are tags on segment values such that initially the segment value
  points to a name which can be resolved at runtime to a mapping to an
  actual live segment -- i.e. in the way dynamic linking works in multics


HYDRA

http://www.cis.upenn.edu/~KeyKOS/

http://www.eros-os.org/eros.html
http://www.coyotos.org/


ATLAS

- first single-level store system?


GOTHIC: https://hal.inria.fr/inria-00075356/file/RR-1202.pdf

SCAP - secure capability architecture, derivative of Cambridge
Capability system

  Karger, Paul Ashley
  Improving Security and Performance for Capability Systems
  PhD thesis, March 1988, Cambridge University.
  http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.64.7003&rep=rep1&type=pdf

  This dissertation examines two major limitations of capability
  systems: an inability to support security policies that enforce
  confinement and a reputation for relatively poor performance when
  compared with non-capability systems.


Apollo AEGIS (and Domain/OS)

https://doc.lagout.org/science/0_Computer%20Science/0_Computer%20History/old-hardware/apollo/Apollo_DOMAIN_Architecture_Feb81.pdf
https://doc.lagout.org/science/0_Computer%20Science/0_Computer%20History/old-hardware/apollo/AEGIS_Overview_1985.pdf
https://doc.lagout.org/science/0_Computer%20Science/0_Computer%20History/old-hardware/apollo/AEGIS_Internals_and_Data_Structures_Jan86.pdf

(via https://filepursuit.com/directory/5128568-AEGIS-Overview-1985-pdf/)


Stratus OpenVOS

https://www.cs.rochester.edu/u/scott/papers/1992_TR418.pdf

	very interesting project to add SLS and proper dynamic linking
	to IRIX, intended for 64-bit systems (see also their Hemlock
	dynamic linking papers)

http://www.greenend.org.uk/rjk/tech/fork.html

[1] http://www.mit.edu/~cbf/thesis.htm

https://lovelaceos.sourceforge.io/  -- unix-like OS in ADA(2012)

Measurements of sharing in Multics
  Warren A. Montgomery
  http://dl.acm.org/citation.cfm?doid=800214.806549

 - this paper notes that a distributed Multics could be feasible (though
   not practical due to memory limitations at the time of writing)

MULTICS and Plan 9, the Big Bangs in Distributed Computing Systems' Universe
  Seyedeh Leili Mirtaheri
  http://drmirtaheri.ir/publication/Journal-papers/a1-1.pdf

IO-Lite:  A Unified I/O Buffering and Caching System
  Vivek S. Pai, Peter Druschel, Willy Zwaenepoel
  https://scholarship.rice.edu/bitstream/handle/1911/17117/1384394.PDF?sequence=1&isAllowed=y
  http://cs.brown.edu/courses/cs161/papers/io-lite.pdf
  [DEAD]http://www.cs.rice.edu/~vivek/IO-Lite/TR97-294.ps

Effects of Data Passing Semantics and Operating System Structure on
Network I/O Performance (1997)
  Jose Carlos Brustoloni
  http://www.cs.pitt.edu/~jcb/papers/thesis.ps
  http://reports-archive.adm.cs.cmu.edu/anon/1997/CMU-CS-97-176.pdf

Towards High-Performance Application-Level Storage Management
  Simon Peter
  https://www.usenix.org/system/files/conference/hotstorage14/hotstorage14-paper-peter.pdf

GEMSOS - Gemini Multiprocessing Secure Operating System for IA32

  - Written in Pascal and leverages IA32 (initially iAPX 286) security
    features to build a highly secure [Class A1 Verified] RTOS kernel
    (architected by Roger Schell)

  - "Designing The GEMSOS Security Kernel" paper mentions that for
    multiprocessor systems they support the concept of "processor-local"
    memory

http://www.helenos.org/	-- portable micro-kernel muli-server (like minix)

Apparently Schiller developed a verifiable kernel prototype on an 11/45
that provided segmented virtual memory implemented at a kernel layer.


M.A.G.I.C.:  Mockapetris And Gregory's Interactive Computer

	https://gunkies.org/wiki/MagicSix


More Taste: Less Greed? (or)
Sending UNIX to the Fat Farm
C H Forsyth (1990???)
http://www.collyer.net/who/geoff/taste.pdf


MenuetOS:

    "MenuetOS is a pre-emptive, real-time and multiprocessor Operating
    System in development for the PC written entirely in 32/64 bit
    assembly language."

	http://www.menuetos.net/