This was one of those, "this would be weird and great", "no it wouldn't" discussions. Sergiy Lozovsky started it up innocently enough, asking how to deal with kernel stack overflow. Then it turned out that I put LISP interpreter inside the Kernel - http://vxe.quercitron.com - It works, but it use a lot of stack memory. It's impossible to rewrite it easily, though I'll investigate why exactly it uses so much of stack memory.

Timothy Miller, who became very outspoken in the thread, wanted to know why on earth Sergiy felt the need to add a Lisp interpreter to the kernel, and Sergiy explained that it was for security purposes. First, it would allow system administrators to configure the system security policies without having to recompile the kernel or write in the lower-level C language; and second, it would protect the system from bugs in that security policy, that might have crept in through errors on the part of the system administrator doing the configuration. As Sergiy explained it, the LISP interpreter is a LISP Virtual Machine (as Java VM). So all bugs are incapsulated and don't affect kernel. Even severe bugs in this LISP kernel module can cause termination of user space application only (except of stack overflow - which I can address). LISP error message will be printed in the kernel log.

There was a pretty big discussion about this, mainly involving folks peppering Sergiy with various objections, and Sergiy attempting to weather them as best as he could. One objection was that the whole kit and kaboodle belonged in user space and not in kernel space. John Stoffel said the entire thing would have to be redesigned in order to have even a small hope of ever getting a foot-hold in the kernel. He pointed to all the protections Sergiy had implemented to keep LISP bugs from harming the kernel, as precisely the reason why the thing belonged in user space in the first place. In user space, there would be no need for the elaborate kernel protections. In response to this, Sergiy argued that he was simply extending the traditional UNIX security model in a natural way; he said that if his LISP interpreter belonged in user space, then the same argument could be used for putting the checking of root privileges and file permissions in user space as well.

Another objection folks made against Sergiy's LISP interpreter was the sheer size of the beast. As Timothy put it in one of his posts, A LISP VM is a big, giant, bloated.... *CHOKE* *COUGH* *SPUTTER* *SUFFOCATE* ... thing which SHOULD NEVER be in the kernel. Sergiy replied that the footprint was just 100K. He hadn't implemented the full Common LISP interpreter, but just a small subset. He said he hadn't been able to find any Virtual Machine that would fit in a smaller space. At this point, Robin Rosenberg pointed out that LeJOS for the Lego Mindstorm would fit into 32K, and was licensed under the Mozilla license.

Another objection was just that LISP was the wrong language in any case, and that, as Timothy put it, system administrators would never write a security policy in LISP, but would use something more like the UNIX shell; or perhaps, he said, Another possibility is to develop a set of tools that compile policies written in C into modules that are loaded/unloaded into the kernel dynamically. :) The compile process would be transparent to the user, because the "insert this policy" tool would run it through GCC (unless the cached .ko was already up to date, etc.).

Overall, Sergiy had interesting responses to all the objections, but it didn't appear that he won anyone over on the important point, namely whether such a thing should go into the kernel or not.

Andy Isaacson said, On modern Linux, apparently the correct way to bypass the buffer cache when writing to a block device is to open the block device with O_DIRECT. This enables, for example, the user to more easily force a reallocation of a single sector of an IDE disk with a media error (without overwriting anything but the 1k "sector pair" containing the error). dd(1) is convenient for this purpose, but is lacking a method to force O_DIRECT. The enclosed patch adds a "conv=direct" flag to enable this usage. Andrew Morton was very encouraging, and asked Andy to do some additional clean-up work as well, while he was busy in that area of the code. While he was working, Andy encountered some difficulties. He said, I can't get O_DIRECT to work on ext3, at all, on 2.4.25 -- open(O_DIRECT) succeeds, but the write returns EINVAL. Andrew replied, ext3 doesn't support O_DIRECT in 2.4 kernels. I did a patch once and I think it's in 2.4-aa kernels. ext3 supports O_DIRECT in 2.6 kernels. Quite a number of filesystems do.

Along the way, it became clear that Andy was only planning to implement his O_DIRECT support for disk writing only, under the assumption that there was no point to have it for disk reading as well. But Andrew said there were the same arguments for enabling O_DIRECT for disk reads as writes, in that it saved CPU usage and avoided over-using the buffer cache. Andy agreed that CPU usage would be lower with O_DIRECT read support, but he felt the gain would not be significant to justify the implementation; while he felt it was the kernel's responsibility to figure out "hey, looks like a streaming read - let's not blow out the buffer cache trying to hold 20GB on a 512M system." If you're saying that the kernel guys have given up and the established wisdom is now "you gotta use O_DIRECT if you don't want to throw everything else out due to streaming data", well... I'm disappointed. And overall, he concluded:

Andrew wasn't swayed by any of this, and said on the contrary:

Andy felt it really wasn't the best idea, but wrote and submitted a patch regardless, according to Andrew's specifications. At around this point, Paul Eggert also posted a patch of his own, to implement the 'dd' features folks had been creating kernel infrastructure for. He said:

    nocreat   do not create the output file
    excl      fail if the output file already exists
    fdatasync physically write output file data before finishing
    fsync     likewise, but also write metadata

    append    append mode (makes sense for output file only)
    direct    use direct I/O for data
    dsync     use synchronized I/O for data
    sync      likewise, but also for metadata
    nonblock  use non-blocking I/O
    nofollow  do not follow symlinks
    noctty    do not assign controlling terminal from file

Jim Meyering from 'coreutils' development said he had applied the patch to the 'coreutils' tree. But Philippe Troin objected, noctty definitely seems overkill... I can't see why dd would ever want to open a file without O_NOCTTY. On systems where O_NOCTTY makes sense (SvR4) that is. Paul agreed that this would just confuse the user, and added a patch to cause 'dd' to always use O_NOCTTY. Jim accepted this patch as well into 'coreutils'.

Andrew Morton announced 2.6.5-mm2, saying:

A small thread followed. Jeremy Higdon got a build error on his 64-bit Intel machine, saying, There have been changes to setup_arch(), including, apparently, the elimination of the cmdline_p argument. Unforunately, that argument was not completely purged from the function. He posted a one-liner to fix compilation on his machine, and Rusty Russell posted a more all-inclusive fix. However, Andrew replied:

Rusty replied:

Alex Williamson said:

The next day he posted again, saying, Here's another approach that's far less ugly than the last and is much more powerful. The code is a little over half the size as a bonus. Rather than specifically poking for certain methods and exposing them, this patch exposes everything. The down side is that all reading and writing of the files need to use binary acpi data structures. This interface certainly provides "shoot yourself in the foot" potential, but the access to the namespace from userspace is hard to beat. Any thoughts on this approach versus the last? This interface and a simple set of libraries to go along with it has a lot of potential. John Belmonte asked, The limitation of this interface is that it's not able to call an ACPI method with some arguments and get the return value, correct? Alex confirmed, Yes, that's unfortunately a limitation. Most of the standard interfaces either take no parameters or have no return value, so they fit nicely into this framework. I'm open to suggestions on how to work around this. We could make the store function save off the method parameters, then the show function would call the method with the saved parameters and return the results. Obviously there are some userspace ordering issues that could make this complicated, but it's easy to code on the kernel side. John gave a pointer to a mailing list thread where the issue was discussed, and Alex replied:

He replied to himself with a patch, saying, I made the assumption that this wasn't that big of a stretch to the bin file interface and extended it to add the arribute pointer. Below is what I came up with. There are luckly only a handful of places using the current sysfs bin file interface, so the additional changes are pretty small. I'm guessing I can probably get rid of the spinlocks I added for the previous implementation, but they're not hurting anything for now. Am I getting closer? Matthew Wilcox replied:

Alex replied, How's this? It behaves the way you described, but might be doing some questionable things with the buffer to get there. Is there a better place to store the return data than back into the buf passed to write() (aka file->private_data)? Without adding callbacks to open/close, I'm not sure how else we can dispose of the results on close. Andy Lutomirski asked why not use an ioctl for this, and Alex replied that according to the thread John gave the pointer to, and according to Matthew, SysFS did not support ioctls at all.

At this point the thread ended.

Andrew Morton announced Linux 2.6.5-mm3, saying:

There were a couple of bug reports and a success report, but no discussion.

Andrew Morton announced the Linux 'merge candidate' 2.6.5-mc4, saying:

There was no reply.

Andrew Morton announced Linux 2.6.5-mm4 (the -mm originally came from the fact that Andrew was working on memory management; it was convenient to keep the name after taking over 2.6 maintenance). He said:

There were various bug reports, including one from Martin Hermanowski, who said, I get an oops when I try to unload the hci_usb module. Greg KH replied, I'm hating that driver right now... There are a number of pending bluetooth patches for that driver that fix a number of different bugs, so I'm leary of trying to see if this is a different one or not at this point in time. Care to apply all of the bluetooth patches and if this still happens, can you report it to the linux-usb-devel and bluez-devel mailing lists? Marcel Holtmann asked which patches Greg meant, saying, There is one from Alan in 2.6.5-mh3 that should fix this problem and I already sent it along with my other fixes (not USB related) to Dave. Greg confirmed that that was the patch he'd meant.

Michael Wu said:

Matt Mackall said, ketchup is a script that automatically patches between kernel versions, downloading and caching patches as needed, and automatically determining the latest versions of several trees. He went on:

He offered an example of ketchup's usage:

 $ ketchup 2.6-mm
 2.6.3-rc1-mm1 -> 2.6.5-mm4
 Applying 2.6.3-rc1-mm1.bz2 -R
 Applying patch-2.6.3-rc1.bz2 -R
 Applying patch-2.6.3.bz2
 Applying patch-2.6.4.bz2
 Applying patch-2.6.5.bz2
 Downloading 2.6.5-mm4.bz2
 Downloading 2.6.5-mm4.bz2.sign
 Verifying signature...
 gpg: Signature made Sat Apr 10 21:55:36 2004 CDT using DSA key ID 517D0F0E
 gpg: Good signature from "Linux Kernel Archives Verification Key
 <ftpadmin@kernel.org>"
 gpg:                 aka "Linux Kernel Archives Verification Key
 <ftpadmin@kernel.org>"
 owner.
 gpg: WARNING: This key is not certified with a trusted signature!
 gpg:          There is no indication that the signature belongs to the
 Primary key fingerprint: C75D C40A 11D7 AF88 9981  ED5B C86B A06A 517D 0F0E
 Applying 2.6.5-mm4.bz2

Zwane Mwaikambo was happy to see this, and asked for an additional feature of support for untarring base trees too so all you'd specify is ketchup 2.6.6 and it'd look for a base tree in $KETCHUP_ARCH if the current working directory isn't a kernel tree.

Gael Le Mignot asked if the linux-wlan-ng driver would make it into the 2.6 kernel. Greg KH replied that AbsoluteValue Systems, the author, had not submitted the code yet for inclusion. Jeff Garzik confirmed that this was true as far as he knew; but he added:

And Jouni Malinen replied, It will be.. WPA2 distracted me from this preparation, but I will try to get back to cleaning up the Host AP driver for 2.6. Actually, I might consider submitting it first without CCMP (AES-counter with CBC-MAC), because that is the part that is likely to take most time to convert for crypto API and this can be easily added separately.

Marcelo Tosatti announced 2.4.26-rc3, saying, It includes few important USB fixes, JFS fixes, amongst others. Geert Uytterhoeven reported an ongoing problem, not solved in 2.4.26-rc3, in which he got warnings of uninitialized variables when compiling JFS into the kernel, using m68k-linux-gcc 2.95.2 and 3.2. There was no discussion about it though.

As part of the ongoing big push to get everything working with SysFS, Michael Hunold had been hard at work adding SysFS support to the DVB Subsystem. However, he wasn't sure he was on the right track. He'd managed to create a /sys/class/dvb/ directory, but he was having trouble creating the subdirectories he wanted under that; he asked for some pointers to documentation. Stephen Hemminger explained, The sysfs directory layout is a logical representation of the underlying data structures in the kernel. Each directory (with exception of attribute groups) has a 1:1 relation ship with a kobject. Greg KH added:

Larry McVoy reported:

Sam Ravnborg reported no problems, although earlier he'd been unable to pull from linux.bkbits.net. Larry said this had been part of the DNS problems that were all fixed; and Andy Isaacson added, Both problems due to an internal network error, which should be fixed now. Thanks for the report; please let us know if you see any further problems.

Randy Dunlap updated his kexec patch for 2.6.5 and 2.6.5, giving locations for the 2.6.4 patches and 2.6.5 patches. There was no discussion.

Greg KH said:

Greg KH posted about 30 patches, and said:

Eric Uhrhane said: