yes - the 'synchronous wakeup' feature is a form of gang scheduling. It in essence uses real process-communication information to migrate 'related' tasks to the same CPU. So it's automatic, no need to declare processes to be part of a 'gang' in some formal (and thus fundamentally imperfect) way.

(another form of 'gang scheduling' can be achieved by binding the 'parent' process to a single CPU - all children will be bound to that CPU as well.)

I was involved in the gangscheduler implementation for the IBM 340 node SP2 cluster at Lawrence Livermore National Lab. Implementation aside, one can show that the total system utilization can be raised from ~60% to a ~90% when doing gang scheduling rather than FIFO scheduling, which one would otherwise do to get a dedicated machine. We got tons of papers on this.

For this it seems sufficient to simply STOP apps on a larger granularity and have that done through a user level daemon. The kernel scheduler simply schedules the runnable threads that given the U-Sched would always amount to a limited number of threads/tasks.

I am glad you brought this up. I'd love to have a generic callback for this. AIX used/has a process change handler that is being called on start/exit.

In Linux, this idea could be done through a generic hook settable through a module... that should be sufficient and would allow for other stuff to be handled as well. For instance in the presence of fast user level communication (e.g. user mapped windows to myrinet the current process could be marked in the communication adapter).

Is Halloween the deadline for submission of patches, or the deadline for inclusion? If I send in reiser4 on Halloween day according to some timezone;-), have I made the deadline for inclusion into 2.6 even if it takes Linus a few months to reach my place in the queue of patches sent to him on Halloween day?

I understand that earlier is better, and I will send it earlier if I can, but even if we do get the reiser4 core (that which does all that V3 does but faster and on top of a plugin infrastructure) done before Halloween, we will inevitably add a few features and tweaks after doing the core, and we will want to send those in at the last minute.

my understanding is that core changes that aren't in by Halloween are not going to be accepted until 2.7. By pre-announcing the deadline, it is hoped that people will have lots of time to submit things that are ready for inclusion, as opposed to rushing to submit when the "freeze" is announced all of a sudden.

If (as we all hope) the important features are added incrementally to the development kernel over the next few months, maybe all of the usage and testing that is going into the development kernel will not be totally lost when the entire kernel is morphed under a huge weight of patches.

It may even mean that there will not be an extra year of features (and bugs) being added to the "frozen" kernel, and we will be able to start 2.7 earlier.

As always, I imagine that as long as you have any core changes in 2.5 before the freeze, it will not be impossible to add self-contained things like filesystems and drivers after the freeze also.

I, in my egocentrism, think it would make more sense to have a deadline for submission rather than a deadline for acceptance, as that would make things predictable for patch submitters, and avoid unintentional overlooking of good patches from obscure persons due to the crush of patches in October.

Pre-announcing the deadline is good, but having it be a deadline on something the patch submitters control (submission time not acceptance time) would be even better.

I would agree, except that this doesn't put any onus on the submitters to get their patches in early, and causes the thundering heard of patches problem the same way that not announcing the patch deadline does.

Note that "accepted" may be a bad term on my part - I can't say if this means that the patch has been recieved by Linus, or whether it actually has to be in the kernel tree at that date.

Note that I wasn't at the kernel summit myself, hence this is all just what I have heard from others.

It's both. We all know Linus doesn't have the time to keep forward-porting our hundreds of patches so he can only include patches into his kernel that apply to the exact same tree he has at that day.

This (and the fact that Linus gets far too much email and patches to look at old ones) is bound to make the Halloween deadline stick for both submission and acceptance.

I hope.

I hope the Halloween feature freeze really will be a feature freeze. Nothing is more frustrating than having a "stable kernel" broken every second release by yet another feature.

If we all restrain ourselves 2.6 will be stable soon and 2.7 will be started shortly after. Backporting "essential" features from 2.7 into a _stable_ 2.6 will be so much easier than trying to stabilise a 2.6-pre that's full to the brim of not-yet-stable new features.

this patch, against current 2.5.27, builds on the patch that let kmem_cache_shrink return the number of pages freed. This value is used as the return value for shrink_dcache_memory and friends.

This is useful not just for more accurate OOM detection, but also as a preparation for putting these reclaimable slab pages on the LRU list. This change was originally done by Ed Tomlinson.

I disagree with the whole approach of having shrink_cache() return the number of pages free.

The number is meaningless, since it has nothing to do with the actual memory zones that are under pressure (right now, the memory zone is almost always ZONE_NORMAL, which is correct, but that's just pure luck rather than anything fundamental).

I'd be much more interested in the "put the cache pages on the dirty list, and have memory pressure push them out in LRU order" approach. Somebody already had preliminary patches.

That gets _rid_ of dcache_shrink() and friends, instead of making them return meaningless numbers.

Side note: while I absolutely think that is the right thing to do, that's also the much more "interesting" change. As a result, I'd be happier if it went through channels (ie probably Andrew) and had some wider testing first at least in the form of a CFT on linux-kernel.

[ Or has it already been in 2.4.x in any major tree? (In which case my testing argument is lessened to some degree and it's mainly just to verify that the forward-port works). ]

I'd suggest that we avoid putting any additional changes into the VM until we have solutions available for:

2: Make it work with pte-highmem (Bill Irwin is signed up for this)

4: Move the pte_chains into highmem too (Bill, I guess)

6: maybe GC the pte_chain backing pages. (Seems unavoidable. Rik?)

Especially pte_chains in highmem. Failure to fix this well is a showstopper for rmap on large ia32 machines, which makes it a showstopper full stop.

If we can get something in place which works acceptably on Martin Bligh's machines, and we can see that the gains of rmap (whatever they are ;)) are worth the as-yet uncoded pains then let's move on. But until then, adding new stuff to the VM just makes a `patch -R' harder to do.

The following patch implements VM strict overcommit for rmap. Strict overcommit couples address space accounting with a strict commit rule to ensure all allocated memory is backed and consequently we never OOM. All memory failures should be pushed into the allocation routines - a page access should never result in a process kill.

The new strict overcommit policies are implemented via sysctl.

This is relatively low-impact on other code and does not change the behavior of the system in the default overcommit policy. Rik has given his approval.

This is based on Alan Cox's work in 2.4-ac with some cleanup and a new overcommit mode for swapless machines. Hugh Dickins also contributed some fixes for shmfs.

Patch is against 2.5.27

The change on the mm/*.c headers for the files changes to include the GPL statement is part of my patch. The code submissions by Red Hat are GPL and the no warranty clauses are applicable. When you resubmit them please include the correct GPL headers I added, or a written guarantee that you will personally take liability for all defects, errors and so on. Also since you changed the code please credit yourself too.

The GPL no warranty clauses were added directly to the file because they are suppsed to be there.

That's a load of bull. They are _NOT_ supposed to be there.

If you want legal disclaimers etc, do them in files you created and you own 100%, not in places that others started and work on. Or put them to the bottom of the file where they aren't in the way. Or add a "read teh GPL in the COPYING file", but don't start adding a ton of crap to core kernel files.

There is no "goodness" in being a lawyer in .c files.

Good. I hate the fact that so many people seem to think that adding 15 lines of copyright notice to a file somehow makes it "more legal". All it does is to give some corporate lawyer a bone, and take up precious real-estate on the first thing you see when you open the file that could be used to actually say what the file does (and who has worked on it).

Slightly off-topic, but in the same vein: I also dislike having tons of changelogs that relate to matters that aren't relevant to the sources any more (because the changes _changed_ them, duh!). The changelogs are valid as a way to show who worked on what, of course, but some people seem to take them to be the beginning of their Great Novel.

I'm hoping that one of the things BK does is to make people less inclined to write change stories in the C files, and more inclined to explain them to me in email when they send the changes in. At which point they are there in a format where you can actually see the "before and after" picture, not just get a feeling that "it looked different before" - well, DUH!

I've created some scripts usefull for doing bitkeeper 2 cvs gateway. I'm currently running them on nl.linux.org (as you can see from the close look). I have not yet figured out how to export resulting CVS to the world.

[These scripts are in CVS at www.sf.net/projects/linux25.]

Hopefully this will be usefull to someone...