16 Dec
2024
16 Dec
'24
9:40 p.m.
> Does anyone know whether there are implementations
of mmap that
> do transparent file sharing? It seems to me that should be possible by
> making the buffer cache share pages with mmapping processes.
These days they all do. The POSIX rationale says:
... When multiple processes map the same memory
object, they can
share access to the underlying data.
Notice the weasel word "can". It is not guaranteed that they will do so
automatically without delay. Apparently each process may have a physically
distinct copy of the data, not shared access to a single location.
The Linux man page mmap(2), for example, makes it very clear that mmap
has a cache-coherence problem, at least in that system. The existence
of msync(2) is a visible symptom of the problem.
[Weasel words of my own: I have not read the POSIX definition of mmap.]
Doug
16 Dec
16 Dec
10:09 p.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
On Mon, Dec 16, 2024 at 10:40 AM Douglas McIlroy <
douglas.mcilroy(a)dartmouth.edu> wrote:
Hmmm — how did that make it through the IEEE editing process? However, that
is not part of the standard if it's in the Rationale - just an explanation
of the standard - which I want want look at more carefully to see what it
says about what is required of mmap(2).
That said, as Heinz and the rest of us who worked on the original version
of what would come to POSIX can tell you, the word "can" was always a
no-no. The word "SHALL" is the the official IEEE word, and if you use
"SHOULD," even in the rationale and it was very much frowned upon. I'll
have try dig up the troff sources to an early draft of the original and do
a "grep -i should * | wc -l" through it. I bet the number is very small
and "can" is zero.
That said, it has been a few years since I have read a current draft, but
less the rationale section (which is not the standard itself).
Clem
Clem
ᐧ
> Does
anyone know whether there are implementations of mmap that
> do transparent file sharing? It seems to me that should be possible by
> making the buffer cache share pages with mmapping processes.
These days they all do. The POSIX rationale
says:
... When multiple processes map the same memory
object, they can
share access to the underlying data.
Notice the weasel word "can". It is not guaranteed that they will do so
automatically without delay. Apparently each process may have a physically
distinct copy of the data, not shared access to a single location.
17 Dec
17 Dec
1:40 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
Clem Cole wrote in
<CAC20D2PfMx=tox0NmLYCi_7345BSr8yoCQZTRVspS8YxAawYdg(a)mail.gmail.com>:
|On Mon, Dec 16, 2024 at 10:40 AM Douglas McIlroy <
|douglas.mcilroy(a)dartmouth.edu> wrote:
|
|>>> Does anyone know whether there are implementations of mmap that
|>>> do transparent file sharing? It seems to me that should be possible by
|>>> making the buffer cache share pages with mmapping processes.
|>
|>> These days they all do. The POSIX rationale says:
|>
|>> ... When multiple processes map the same memory object, they can
|>> share access to the underlying data.
|>
|> Notice the weasel word "can". It is not guaranteed that they will do so
|> automatically without delay. Apparently each process may have a physical\
|> ly
|> distinct copy of the data, not shared access to a single location.
|>
|Hmmm — how did that make it through the IEEE editing process? However, that
|is not part of the standard if it's in the Rationale - just an explanation
|of the standard - which I want want look at more carefully to see what it
|says about what is required of mmap(2).
|
|That said, as Heinz and the rest of us who worked on the original version
|of what would come to POSIX can tell you, the word "can" was always a
|no-no. The word "SHALL" is the the official IEEE word, and if you use
|"SHOULD," even in the rationale and it was very much frowned upon.
I'll
They are all lowercase nowadays.
|have try dig up the troff sources to an early draft of the original and do
|a "grep -i should * | wc -l" through it. I bet the number is very small
|and "can" is zero.
They offer lots of hints like "can be implemented efficiently" or
"conforming implementations cannot count on", which turns "can"
into a "dying butterfly". (In the rationale's.)
|That said, it has been a few years since I have read a current draft, but
|less the rationale section (which is not the standard itself).
The 2024 version is 4057 pages, all in all. It would -- in my
opinion -- make a nice discussion whether it would be better for
anybody, including engineers, to take the time to read that amount
of cultural foundation, real literature that is, instead of
a technical standard. (I admit i personally perform standard
hopping and have at times some spare time for other things,
because if i fail it currently endangers noone for one, and when
i fall i stand up anyway, unless i do no more, but then having
lived in mindsets++ of fantastic literats.)
|Clem
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
|
|And in Fall, feel "The Dropbear Bard"s ball(s).
|
|The banded bear
|without a care,
|Banged on himself for e'er and e'er
|
|Farewell, dear collar bear
1:47 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
Steffen Nurpmeso wrote in
<20241216194043.PqDb-or7(a)steffen%sdaoden.eu>:
|Clem Cole wrote in
| <CAC20D2PfMx=tox0NmLYCi_7345BSr8yoCQZTRVspS8YxAawYdg(a)mail.gmail.com>:
||On Mon, Dec 16, 2024 at 10:40 AM Douglas McIlroy <
||douglas.mcilroy(a)dartmouth.edu> wrote:
||
||>>> Does anyone know whether there are implementations of mmap that
||>>> do transparent file sharing? It seems to me that should be possible by
||>>> making the buffer cache share pages with mmapping processes.
||>
||>> These days they all do. The POSIX rationale says:
||>
||>> ... When multiple processes map the same memory object, they can
||>> share access to the underlying data.
||>
||> Notice the weasel word "can". It is not guaranteed that they will do so
||> automatically without delay. Apparently each process may have a \
||> physical\
||> ly
||> distinct copy of the data, not shared access to a single location.
I think it was in FreeBSD when sed(1) once got an optimization to
mmap(1) data, but it was reverted because of crashes caused by
concurrent modifications. (There definetely was something like
this, but FreeBSD / sed is nothing but what i remember.)
But there is no weasel word in the real standard text, but lots of
"shall". The only "can" there are
There may be implementation-defined limits on the number of
memory regions that can be mapped (per process or per system).
and
If such a limit is imposed, whether the number of memory regions
that can be mapped by a process is decreased by the use of
shmat( ) is implementation-defined.
which are no "weasel-can's".
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
|
|And in Fall, feel "The Dropbear Bard"s ball(s).
|
|The banded bear
|without a care,
|Banged on himself for e'er and e'er
|
|Farewell, dear collar bear
2:58 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
On Mon, Dec 16, 2024 at 2:40 PM Steffen Nurpmeso <steffen(a)sdaoden.eu> wrote:
They are all lowercase nowadays.
They were always case-sensitive to use. I used upper case to make it stand
out in my message.
|have try dig up the troff sources to an early draft of the original and
do
|a "grep -i should * | wc -l" through it. I bet the number is very small
|and "can" is zero.
They offer lots of hints like "can be implemented efficiently" or
"conforming implementations cannot count on", which turns "can"
into a "dying butterfly". (In the rationale's.)
It always say. *"conforming implementations shall not count on."*
The standard has to be as precise as it can be. If there are
implementation grey areas, then the standard needs to state that too.
Again - we were careful when we wrote original documents (and had lots of
arguments as you can imagine).
As for what you read, I'm not worried. The fact the over time, standards
stop being what they were originally intended and take a life of their
own. POSIX is less of an issue than say, C++ or even FORTRAN for that
matter. But over time, a new group of people want to "make their mark."
Some of us grew tired of arguing. We got what we originally set out to do,
and I left the POSIX work soon after the *.2 began. I worked through one
draft of it, but was not there for approval, although I helped with *.4 at
one point.
Someone implementing something new, be it a language or a system, should be
familiar with the standards. They is a lot be learned both good and bad.
Reinventing things, particularly bad ideas, is hardly for the greater good
(although the implementor might find it fun). e.g. C++, IMO, is a great
example of what >>not<< to do. The core POSIX.1 interface, think is
excellent and gets to the point. After that, YMMV and the value you get
from it also is a bit variable. But thinking the world should be stagnant
and believing that C or UNIX (or modern Unix, a.k.a. Linux) is "the end" is
hardly a good idea either.
So, reading and learning about them does have some merit. The question is
how much and how far to go.
Clem
ᐧ
3:50 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
Clem Cole wrote in
<CAC20D2OzCzf=zpGaPzNW-_tF6AEsw-EWc8kGnL_usYLuvdCh0A(a)mail.gmail.com>:
|On Mon, Dec 16, 2024 at 2:40 PM Steffen Nurpmeso <steffen(a)sdaoden.eu> \
|wrote:
|> They are all lowercase nowadays.
|>
|They were always case-sensitive to use. I used upper case to make it stand
|out in my message.
Ah, ok. In the last years i am reading lots of IETF outcome, and
for them this makes a world of a difference.
|>|have try dig up the troff sources to an early draft of the original and
|> do
|>|a "grep -i should * | wc -l" through it. I bet the number is very
small
|>|and "can" is zero.
|>
|> They offer lots of hints like "can be implemented efficiently" or
|> "conforming implementations cannot count on", which turns
"can"
|> into a "dying butterfly". (In the rationale's.)
|>
|It always say. *"conforming implementations shall not count on."*
|The standard has to be as precise as it can be. If there are
|implementation grey areas, then the standard needs to state that too.
|
|Again - we were careful when we wrote original documents (and had lots of
|arguments as you can imagine).
|
| As for what you read, I'm not worried. The fact the over time, standards
|stop being what they were originally intended and take a life of their
|own. POSIX is less of an issue than say, C++ or even FORTRAN for that
|matter. But over time, a new group of people want to "make their mark."
..i would even put on top that if someone does *not* want to do
that, they react irritated. "Too many marks", at least in the
IETF world.
|Some of us grew tired of arguing. We got what we originally set out to do,
|and I left the POSIX work soon after the *.2 began. I worked through one
|draft of it, but was not there for approval, although I helped with *.4 at
|one point.
|
|Someone implementing something new, be it a language or a system, should be
|familiar with the standards. They is a lot be learned both good and bad.
|Reinventing things, particularly bad ideas, is hardly for the greater good
|(although the implementor might find it fun). e.g. C++, IMO, is a great
Wise words of an, please excuse that, older man. It is surely
nothing but true, but it will not quell the overflowing hormons of
the young warriors, nor false prowd, nor .. etc etc etc. I do not
know. One must not prevent young people from making their own
experiences for sure, at best they can be guided a bit to not run
into fatality dead-ends, maybe.
|example of what >>not<< to do. The core POSIX.1 interface, think is
|excellent and gets to the point. After that, YMMV and the value you get
|from it also is a bit variable. But thinking the world should be stagnant
|and believing that C or UNIX (or modern Unix, a.k.a. Linux) is "the end" is
|hardly a good idea either.
The holy men of India even do nothing at all! Except walking to
the fountain of the holy river. And already in sight, the water
drag dies into the ocean. You know, i have no idea regarding
technology, it surely will iterate further. Surely the computer
of for example space odysee was created after long talks with wise
men, in the spirit of Lidlicker etc. The big picture.
But regarding "the end", i for myself do not like the Rust
programning language, regardless what you say.
|So, reading and learning about them does have some merit. The question is
|how much and how far to go.
It is surely more than only merit. To the opposite, one should
not drown in deep respect aka be slained due to the high
intellectual penetration of a document like the POSIX standard,
which is in the works for fourty years and has seen hundreds of
highly educated people. (And at much later times, the one or the
other idiot.)
I, by the way, would expect the traditional answer being "learn
the abc/koran/calligraphy/.. studiously while you are young, then
gain the mastership and pleasure through living experience when
you are older". As well as, and that is very important in my
humble opinion, "if it doesn't come naturally, leave it".
Ie, books are one thing, living it is another.
--steffen
|
|Der Kragenbaer, The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
|
|And in Fall, feel "The Dropbear Bard"s ball(s).
|
|The banded bear
|without a care,
|Banged on himself for e'er and e'er
|
|Farewell, dear collar bear
1:02 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
On Mon, Dec 16, 2024 at 10:40:34AM -0500, Douglas McIlroy wrote:
I don't see this language in the 2024 version of POSIX (IEEE Std 1003.1-2024).
> Does
anyone know whether there are implementations of mmap that
> do transparent file sharing? It seems to me that should be possible by
> making the buffer cache share pages with mmapping processes.
These days they all do. The POSIX rationale
says:
> ... When multiple processes map the same memory object, they can
> share access to the underlying data. Notice the weasel word "can". It is not
guaranteed that they will do so
automatically without delay. Apparently each process may have a physically
distinct copy of the data, not shared access to a single location.
The Linux man page mmap(2), for example, makes it very clear that mmap
has a cache-coherence problem, at least in that system. The existence
of msync(2) is a visible symptom of the problem.
Huh? What language are you talking about? msync(2) is about when
dirty data is written to stable storage (e.g., written back out to
disk). It works much like fsync(2).
There is a cache-coherency problem between what's in the buffer/page
cache (Linux has a unified page cache, so file-backed blocks are not
cached in the buffer cache; only the page cache) and O_DIRECT. Some
OS's will tell you that there is no coherency guarantees at all.
Linux will give a best-efforts attempt at coherency which is to say
that before doing an O_DIRECT read, any dirty pages will be written
out to disk before the O_DIRECT read is allowed to proceed, and before
doing an O_DIRECT write, any underlying pages in the page cache will
be invalidated. It is not 100% guaranteed to be coherent, and system
which try to mix simultaneous buffered and O_DIRECT I/O may not
necessarily get what they want.
However, if what you are talking about is multiple processes mmap'ing
the same file, they *will* get the same page in the page cache mapped
into the page tables, which means that writes by one process will be
immediately seen by another process (modulo CPU cache coherency
guarantees, of course; Intel is pretty good; with a weekly consistent
architecture like Alpha, your mileage may vary unless you know what
you are doing and write your assembly code accordingly).
Cheers,
- Ted
1:08 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
It appears that Douglas McIlroy <douglas.mcilroy(a)dartmouth.edu> said:
[Weasel words of my own: I have not read the POSIX
definition of mmap.]
We're in luck, I have it right here. On pages 529-530:
2.8.3.2 Memory Mapped Files
Range memory mapping operations are defined in terms of pages. Implementations may
restrict the size and alignment of range mappings to be on page-size boundaries. The page
size,
in bytes, is the value of the configurable system variable {PAGESIZE}. If an
implementation has
no restrictions on size or alignment, it may specify a 1-byte page size.
Memory mapped files provide a mechanism that allows a process to access files by directly
incorporating file data into its address space. Once a file is mapped into a process
address space,
the data can be manipulated as memory. If more than one process maps a file, its contents
are
shared among them. If the mappings allow shared write access, then data written into the
memory object through the address space of one process appears in the address spaces of
all
processes that similarly map the same portion of the memory object.
The mmap() man page on debian says this, which seems clear enough:
MAP_SHARED
Share this mapping. Updates to the mapping are visible to other processes mapping the
same region, and (in the case of file-backed mappings) are carried through to
the underlying file. (To precisely control when updates are carried through to the
underlying file requires the use of msync(2).)
The msync() call is a minor fudge that lets you control how briskly changes are
written back to disk. But it seems clear enough that all the processes that have
a page mapped see the same page, and msync is for databases that want to bs sure
that changes are committed to permananent storage (a term in the POSIX spec)
before unlocking or unmapping.
R's,
John
PS: I can believe there are some versions of linux that screwed up disk cache
coherency, but that just means they don't properly implement the spec, not for
the first time. I mean, it's not *that* hard to make all the maps point to the
same physical page frame, even on a machine like POWER with reverse page maps.
1:45 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
On 12/16/24 2:08 PM, John Levine wrote:
It appears that Douglas McIlroy
<douglas.mcilroy(a)dartmouth.edu> said:
Here are the current (POSIX.1-2024) versions of the appropriate definitions
and the mmap() description, for those who want to read along.
https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#ta…
https://pubs.opengroup.org/onlinepubs/9799919799/functions/mmap.html#tag_17…
[Weasel words of my own: I have not read the
POSIX definition of mmap.]
We're in luck, I have it right here. On pages 529-530:
2.8.3.2 Memory Mapped Files
Is that from the Rationale? Here's the current version.
https://pubs.opengroup.org/onlinepubs/9799919799/xrat/V4_xsh_chap01.html#ta…
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
4:43 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
It appears that Chet Ramey via TUHS <chet.ramey(a)case.edu> said:
No, like I said it's on pages 529-530 of the PDF version of IEEE 1003.1-2024
which I downloaded from the IEEE.
Your web site has a copy here
https://pubs.opengroup.org/onlinepubs/9799919799/functions/V2_chap02.html#t…
R's,
John
-=-=-=-=-=-
-=-=-=-=-=-
On 12/16/24 2:08 PM, John Levine wrote:
Well, that's confusing. That web site seems to have the same text as the
IEEE's PDF but organized differently.
It appears that Douglas McIlroy
<douglas.mcilroy(a)dartmouth.edu> said:
Here are the current (POSIX.1-2024) versions of the appropriate definitions
and the mmap() description, for those who want to read along. [Weasel words of my own: I have not read the
POSIX definition of mmap.]
We're in luck, I have it right here. On pages 529-530: 2.8.3.2 Memory
Mapped Files
Is that from the Rationale?
5:19 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
On 12/16/24 5:43 PM, John Levine wrote:
Sure, it's the Open Group. The advantage is it's public and on the web.
(Plus I always have a browser tab open on it.)
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRU chet(a)case.edu http://tiswww.cwru.edu/~chet/
Here are the
current (POSIX.1-2024) versions of the appropriate definitions
and the mmap() description, for those who want to read along.
Well, that's confusing. That web site seems to have the same text as the
IEEE's PDF but organized differently.
4:19 a.m.
New subject: mmap, was SCCS, TeamWare, BitKeeper, and Git
On Dec 16, 2024, at 7:40 AM, Douglas McIlroy <douglas.mcilroy(a)dartmouth.edu> wrote:
May be this has to do with using POSIX on multiprocessor systems that
are *not* cache coherent? May be less of an issue for now[1] though but
weren't there such systems in the past?
In any case shared access by itself is not enough for proper operation.
Bakul
[1] Given very high clock rates and increasing # of cores, I wonder
how long before they start using CSMA/CD for coherency protocols!
... When multiple processes map the same memory
object, they can
share access to the underlying data.
Notice the weasel word "can". It is not guaranteed that they will do so
automatically without delay. Apparently each process may have a physically
distinct copy of the data, not shared access to a single location.
5
days inactive
5
days old
11 comments
7 participants
participants (7)
-
Bakul Shah -
Chet Ramey -
Clem Cole -
Douglas McIlroy -
John Levine -
Steffen Nurpmeso -
Theodore Ts'o