The original Docker (and the current Podman) created each layer as an overlay filesystem. So each layer was essentially an ephemeral container. If a build failed, you could actually just run the last successful layer with a shell and see what's wrong.
More importantly, the layers were represented as directories on the host system. So when you wanted to run something in the final container, Docker just needed to reassemble it.
Buildkit has broken all of it. Now building is done, essentially, in a separate system, the "docker buildx" command talks with it over a socket. It transmits the context, and gets the result back as an OCI image that it then needs to unpack.
This is an entirely useless step. It also breaks caching all the time. If you build two images that differ only slightly, the host still gets two full OCI artifacts, even if two containers share most of the layers.
It looks like their Bazel infrastructure optimized it by moving caching down to the file level.
C# I don’t believe was/is a common tuning for most western instruments, classical or modern.
A digital piano can transpose things to make it “easier” to play.
Cursory google search says that a sitar is traditionally tuned to something useful for c#
I’m curious if C# is one of those notes that lines up nicely with whatever crappy consumer stereos/subs were capable of reasonable reproducing in the 90s as electronic music was taking off and it stuck around as a tribal knowledge for getting more “oomph” out of your tracks.
Sam Altman has made a lot of grandiose claims about how much power he's going to need to scale LLMs, but the evidence seems to suggest the amount of power required to train and operate LLMs is a lot more modest than he would have you believe. (DeepSeek reportedly being trained for just $5M, for example.)
I saw a claim that DeepSeek had piggybacked off of some aspect of training that ChatGPT had done, and so that cost needed to be included when evaluating DeepSeek.
This training part of LLMs is still mostly Greek to me, so if anyone could explain that claim as true or false and the reasons why, I’d appreciate it
I think the claim that DeepSeek was trained for $5M is a little questionable. But OpenAI is trying to raise $100B which is 20,000 times as much as $5M. Though even at $1B I think it's probably not that big a deal for Google or OpenAI. My feeling is they can profit on the prices they are charging for their LLM APIs, and that the dominant compute cost is inference, not training. Though obviously that's only true if you're selling billions of dollars worth of API calls like Google and OpenAI.
OpenAI has had $20B in revenue this year, and it seems likely to me they have spent considerably less than that on compute for training GPT5. Probably not $5M, but quite possibly under $1B.
So LLMs predict the next token. Basically, you train them by taking your training data that's N words long and, for X = 1 to N, and optimizing it to predict token X using tokens 1 to X-1.
There's no reason you couldn't generate training data for a model by getting output from another model. You could even get the probability distribution of output tokens from the source model and train the target model to repeat that probability distribution, instead of a single word. That'd be faster, because instead of it learning to say "Hello!" and "Hi!" from two different examples, one where it says hello and one where it says hi, you'd learn to say both from one example that has a probability distribution of 50% for each output.
Sometimes DeepSeek said it's name is ChatGPT. This could be because they used Q&A pairs from ChatGPT for training or because they scraped conversations other people posted where they were talking to ChatGPT. Or for unknown reasons where the model just decided to respond that way, like mixing up some semantics of wanting to say "I'm an AI" and all the scraped data referring to AI as ChatGPT.
Short of admission or leaks of DeepSeek training data it's hard to tell. Conversely, DeepSeek really went hard into an architecture that is cheap to train, using a lot of weird techniques to optimize their training process for their hardware.
Personally, I think they did. Research shows that a model can be greatly improved with a relatively-small set of high quality Q&A pairs. But I'm not sure the cost evaluation should be influenced that much, because the ChatGPT training price was only paid once, it doesn't have to be repaid for every new model that cribs its answers.
We're paying for pyx. Wouldn't have if we didn't enjoy enjoy uv and ruff.
It's definitely a narrow path for them to tread. Feels like the best case is something like Hashicorp, great until the founders don't want to do it anymore.
> Feels like the best case is something like Hashicorp
Wow, that's probably my go-to case of things going south, not "best case scenario". They sold to IBM, a famous graveyard for software, and on the way there changed from FOSS licensing to their own proprietary ones for software the community started to rely on.
You're not wrong, but a) most of the badness happened after the founders checked out and b) it's hard to find examples of developer tool companies doing better.
You however, are. Hashimoto didn't leave until December 2023, Hashicorp announced the license change August 10, 2023. Also way back in September 2021 they started having staffing issues and stopped accepting community contributions, and also made the questionable choice of going public that same year.
You might be on to something with point B, hard to find good examples of developer tool companies that don't eventually turn sour. However, there are countless examples of successful and still very useful developer tools out there, maybe slapping a company on it and sell a "pro" version isn't the way to go?
I actually would argue that Hashimoto "left" earlier. He "stepped down" from the executive team July 2021 and became an individual contributor then. He likely lost interest/power a long time before 2023.
Meh. The end of the company many of us admire was a combination of the founders giving up control to the usual villains and the venture business model failing for developer tools. I don't think the specific departure date matters very much; things started to degrade earlier.
As for "slapping a company on it", I agree, but also I don't think we've developed a viable alternative. Python has been limping along with one toolchain or another for my entire career (multiple decades) and it took Astral's very specific approach to create something better. It's fair to ask why they needed to be venture backed, but they clearly are and the lack of successful alternatives is telling.
My issue with them is that they claim their tools replace existing tools, but they don't bother to actually replicate all of the functionality. So if you want to use the full functionality of existing tools, you need to fall back on them instead of using Astral's "replacements". It's like one step forward and one step back. For me personally, speed of the tooling is not as important as what the tooling can check, which is very important for a language like Python that is very easy to get wrong.
If there are specific incompatibilities or rough edges you're running into, we're always interested in hearing about them. We try pretty hard to provide a pip compatibility layer[1], but Python packaging is non-trivial and has a lot of layers and caveats.
Is there any plan for a non-“compatibility layer” way to do anything manual or nontrivial? uv sync and uv run are sort of fine for developing a distribution/package, but they’re not exactly replacements for anything else one might want to do with the pip and venv commands.
As a very basic example I ran into last week, Python tooling, even the nice Astral tooling, seems to be almost completely lacking any good detection of what source changes need to trigger what rebuild steps. Unless I’ve missed something, if I make a change to a source tree that uv sync doesn’t notice, I’m stuck with uv pip install -e ., which is a wee bit disappointing and feels a bit gross. I suppose I could try to put something correct into cache-keys, but this is fundamentally wrong. The list of files in my source tree that need to trigger a refresh is something that my build system determines when it builds. Maybe there should be a way to either plumb that into uv’s cache or to tell uv that at least “uv sync” should run the designated command to (incrementally) rebuild my source tree?
(Not that I can blame uv for failing to magically exfiltrate metadata from the black box that is hatchling plus its plugins.)
> Is there any plan for a non-“compatibility layer” way to do anything manual or nontrivial?
It's really helpful to have examples for this, like the one you provide below (which I'll respond to!). I've been a maintainer and contributor to the PyPA standard tooling for years, and once uv "clicked" for me I didn't find myself having to leave the imperative layer (of uv add/sync/etc) at all.
> As a very basic example I ran into last week, Python tooling, even the nice Astral tooling, seems to be almost completely lacking any good detection of what source changes need to trigger what rebuild steps.
Could you say more about your setup here? By "rebuild steps" I'm inferring you mean an editable install (versus a sdist/bdist build) -- in general `uv sync` should work in that scenario, including for non-trivial things where e.g. an extension build has to be re-run. In other words, if you do `uv sync` instead of `uv pip install -e .`, that should generally work.
However, to take a step back from that: IMO the nicer way to use uv is to not run `uv sync` that much. Instead, you can generally use `uv run ...` to auto-sync and run your development tooling within an environment than includes your editable installation.
By way of example, here's what I would traditionally do:
python -m venv .env
source .env/bin/activate
python -m pip install -e .[dev] # editable install with the 'dev' extra
pytest ...
# re-run install if there are things a normal editable install can't transparently sync, like extension builds
Whereas with uv:
uv run --dev pytest ... # uses pytest from the 'dev' dependency group
That single command does everything pip and venv would normally do to prep an editable environment and run pytest. It also works across re-runs, since it'll run `uv sync` as needed under the hood.
My setup is a mixed C/C++/Python project. The C and C++ code builds independently of the Python code (using waf, but I think this barely matters -- the point is that the C/C++ build is triggered by a straightforward command and that it rebuilds correctly based on changed source code). The Python code depends on the C/C++ code via ctypes and cffi (which load a .so file produced by the C/C++ build), and there are no extension modules.
Python builds via [tool.hatch.build.targets.wheel.hooks.custom] in pyproject.toml and a hatch_build.py that invokes waf and force-includes the .so files into useful locations.
Use case 1: Development. I change something (C/C++ source, the waf configuration, etc) and then try to run Python code (via uv sync, uv run, or activating a venv with an editable install). Since there doesn't seem to be a way to have the build feed dependencies out to uv (this seems to be a deficiency in PEP 517/660), I either need to somehow statically generate cache-keys or resort to reinstall-package to get uv commands to notice when something changed. I can force the issue with uv pip install -e ., although apparently I can also force the issue with uv run/sync --reinstall-packages [distro name]. [0] So I guess uv pip is not actually needed here.
It would be very nice if there was an extension to PEP 660 that would allow the editable build to tell the front-end what its computed dependencies are.
Use case 2: Production
IMO uv sync and uv run have no place in production. I do not want my server to resolve dependencies or create environments at all, let alone by magic, when I am running a release of my software built for the purpose.
My code has, long before pyproject.toml or uv was a thing and even before virtual environments existed (!), had a script to build a production artifact. The resulting artifact makes its way to a server, and the code in it gets run. If I want to use dependencies as found by uv, or if I want to use entrypoints (a massive improvement over rolling my own way to actually invoke a Python program!), as far as I can tell I can either manually make and populate a venv using uv venv and uv pip or I can use UV_PROJECT_ENVIRONMENT with uv sync and abuse uv sync to imperatively create a venv.
Maybe some day uv will come up with a better way to produce production artifacts. (And maybe in the distant future, the libc world will come up with a decent way to make C/C++ virtual environments that don't rely on mount namespaces or chroot.)
[0] As far as I can tell, the accepted terminology is that the thing produced by a pyproject.toml is possibly a "project" or a "distribution" and that these are both very much distinct from a "package". I think it's a bit regrettable that uv's option here is spelled like it rebuilds a _package_ when the thing you feed it is not the name of a package and it does not rebuild a particular package. In uv's defense, PEP 517 itself seems rather confused as well.
uv needs to support creation of zipapps, like pdm does (what pex does standalone).
Various tickets asking for it, but they also want to bundle in the python interpreter itself, which is out of scope for a pyproject.toml manager: https://github.com/astral-sh/uv/issues/5802
Their integration with existing tools seems to be generally pretty good.
For example, uv-build is rather lacking in any sort of features (and its documentation barely exists AFAICT, which is a bit disappointing), but uv works just fine with hatchling, using configuration mechanisms that predate uv.
(I spent some time last week porting a project from an old, entirely unsupportable build system to uv + hatchling, and I came out of it every bit as unimpressed by the general state of Python packaging as ever, but I had no real complaints about uv. It would be nice if there was a build system that could go even slightly off the beaten path without writing custom hooks and mostly inferring how they’re supposed to work, though. I’m pretty sure that even the major LLMs only know how to write a Python package configuration because they’ve trained on random blog posts and some GitHub packages that mostly work — they’re certainly not figuring anything out directly from the documentation, nor could they.)
Getting from 95% compatible to 100% compatible may not only take a lot of time, but also result in worsening the performance. Sometimes it's good to drop some off the less frequently used features in order to make the tool better (or allow for making the tool better)
Damn it, this unicorn farting rainbows and craping gold is not yet capable of towing another car. I don't know why they advertise it as a replacement for my current mode of transportation.
COVID was in many ways the opposite of a reset. It further solidified massive wealth disparity and power concentration. The positive feedback loop doesn’t appear to have a good off-ramp.
The off ramp to the loop was always a crisis like a recession or war. Covid could have been such an example. If the government hadn't bailed everyone out, the pandemic would have been a mass casualty event for companies and their owners. The financial crisis of 2008 also would have been such an event if not for the bailouts. The natural state of the economy is boom and bust, with wealth concentration increasing during the booms and dropping during the busts. But it turns out the busts are really painful for everyone, and so our government has learned how to mostly keep them from happening. Turns out people prefer the ever increasing concentration of wealth an power that this results in over the temporary pain of economic depression.
At today's prices perhaps, but pre ChatGPT you just have to run more of it + more error correction. Not great for the power budget but not anything significant in the grand scheme of things.
You need parity, which is cheap, or lockstep duplexing, which isn't. Or, you know, sometimes you can just restart malfunctioning processes and repair corrupted filesystems while you run the failed tasks again on another node.
Well the argument some of these companies are making is that it would be cheaper over 10 years (some things like power can be cheaper in space, and you can get it from solar nearly 24h a day). It seems likely to me (as it does many other people) that it won't be cheaper, but if it's the same price or mildly more expensive there might be a regulatory incentive to train a ML model in space instead of a place like the EU
reply