I have this opinion about no other language. I am generally neutral about languages I don't use. But literally my only experience with Python is porting to Python 3. It's like a sitcom scenario of how to force the worst possible first impression for a language ever. The amount of slowdown and pain that Python 3 has generated on our non-Python project is really just out of this world ridiculous.

I think you are missing the key point here: I don't write a ton of Python. I don't write any Python. Of course you think this is old news. You write a lot of Python. You're totally plugged in. You've known this is coming, this inside baseball is "the thing" for you. My point is that, believe it or not, the rest of the world does not keep up to date with whatever the Python community is doing. This is not unique to Python. Most people in dev community X are not aware of what is happening in dev community Y. What is unique to Python is that all of a sudden we were forced to become aware as the downstream apps began to break. By creating a backwards compatibility issue in an interpreted language, you necessarily "backload" the end-user pain to when the apps, which by definition are updated later than the code within the apps, begin to break as packages are removed from package managers, or things are EOL'ed, etc. This is a situation that I can honestly not remember happening with any other language. I can't remember any other circumstance, and certainly not one where it proceeded to happen repeatedly, where I've been abruptly made aware that utilityX was written in a no-longer-supported version of C/perl/whatever (when I previously didn't even know what language it was), after I had been running it for ages.

A saw still works after a decade on a shelf, programs should still work too because they are tools.

I can't think of any 100-year old software that is being used today (ha-ha).

I'm sure some airline reservation systems and banking systems written in COBOL in the 70s are still around, but they all have ongoing maintenance to keep them working correctly.

The commenter's complaint seems particularly worse with dynamic languages like JavaScript and Python than compiled things. Old projects written in node are often just as hard to get running or installed as something written in Python 2. The dependencies are out of date, you have to install an older version of Node.js, or the dependencies just don't exist anymore. And JS stuff usually generates a huge tree of dependencies even if they only directly rely on a handful of packages.

End of story, I think if these Python-2 tools/scripts were such a huge part of his project that they keep breaking builds, the company should be invest time in either rewriting them in their language of choice or maintaining those projects or a fork of those projects.

One of my old bosses was strict about this - no COMPANY projects should be relying on external package repositories, unapproved third-party dependencies, definitely not some github project maintained by only a few people. Yeah, I think if your company is relying on "requests" or some other massively popular project with tons of community maintenance, you don't want to fork that and it's important to use the standard versions. But if you're bringing in something that is used and maintained by only a few people - you either need to become part of the community that maintains it publicly or fork the project and maintain your own version.

This isn't NIH syndrome, it's basic protocol for keeping your builds reliable and dependency management simple.

Why should I, an end user, ever need to build WikidPad? I don't need to do so on Windows, why doesn't Linux support the same thing, shipping an executable?

When you go to the store, you're not presented with a set of tools for making a saw, you're given a complete, ready to go tool. Perhaps the very idea of users having to build things in Linux is the problem?

It is very similar to the problems with Windows, always pushing updates/upgrades that aren't necessary, and often break things. Windows machines are far less reliable on Wednesday than they are on Monday, because of Microsoft's idea of pushing patches on Tuesdays.

Truly a hacker news experience.

Also, I am not "trying hard" not to care about Python. I specifically said in my very first comment I go out of my way to avoid it now. I think that falls in the "cares" category. My default state is to not be aware of what language things are written in. I don't know if some given tool is written in C, bash, or JS. If I were to coincidentally learn what language it was written in, it might serve as a curiosity, but not really phase me either way. However, I was forced to become aware of a bunch of Python stuff since it started breaking, and then forced to learn about all the weird workarounds to get it working. Given that I was forced to repeatedly deal with dependency hell with apps in this language, I am now choosing to avoid it if I can in the future if I learn that an app is written in it.

The "magic date" when the breaking update came was 15 years ago. Python gave this developer 10 years to get with the program and update to the new version, and Debian kept it alive for an additional 5 just in case.

If a developer can't get their act together to change over their codebase to preserve minimum functionality despite FIFTEEN YEARS of warning, I don't want to be using their app because who knows what other obvious problems they are ignoring.

It’s called open source entitlement. The kind of people who fill up your issues and don’t push PRs and only donate with their melodramatic keyboard.

Nobody should ever have to be hit with obscure pip/virtualenv/pypy/etc bugs for trying to use your command line app. Nobody should have to suffer through an “installation” step that requires installing the language itself and running a language-provided package manager to install dependencies/etc. I don’t care how much you wrap it in a setup.sh that tries (and typically, fails miserably) to automate everything.

(For OS’s like linux, your distro-provided package managers make the situation slightly better, but only if you ship your app as a package using these distro’s built-in package systems, which is a huge pain if you want to support multiple distros.)

But if you opt to skip the package manager and want to ship a “simple” installation experience using something like a setup.sh, and you want to use a scripting language like python/ruby/etc (integrated with an installation of that language in $HOME or in some system directory), you’re treating your users to a world of pain. Please, please use a compiled language, and ship a binary (or collection of binaries/libraries that can sit in a folder together and just work when you run it.)

That is, I believe, the entire point of the parent, is it not? That this whole migration was not quarantined to the Python developer community. That it spilled over to users in totally unrelated spaces.

That was it. Since then I use Git everywhere even though I prefer Hg.

I feel your response is best summarized as: "I like Python. I've never had this problem and don't think it's a legitimate problem. Only $megacorp users have this issue, and their concerns aren't important."

If you dismiss the problem (as you've done) then you don't have to intellectually address it. The Python 2 -> 3 transition was hard for those that huge productive workloads in Python2, and this pain will have a long tail.

Whoever missed that is self responsible for all self harm done by ignoring that.

However that's not what you're complaining about. You are complaining about is that you want others to support python2 forever, just as you may want browsers/sites to support `alert()`, the `<blink>` tag and IE 6.0 forever. However that's not how the world works.

If you wish to use something nobody else wants to support, then you need to support it yourself.

Otherwise, take a moment to understand the thing you are using and follow any instructions to upgrade. Don't put ignorance on a pedestal and then make the argument that the fruits of your ignorance are only the fault of others and not yourself.

Incorrect. You have to understand that I am not a Python user, and don't a priori care what they do with the language. I only become annoyed when something breaks on its own, or if changing one unrelated thing breaks other unrelated things. This happens for a variety of reasons:

1. Either date-based breaking, or patch releases designed just to throw up deprecations, and breaking the tools you actually use.

2. Packages and or binaries disappearing from package managers, such that even if I have a pinned version and I make no changes, an accidental cache invalidation can now break the entire image.

3. The fact that updating one utility may break others (either because you need to figure out this python symlinking issue or whatever).

4. Python causing discontinuities that don't exist in other bundled build tools. Going from buildpacks:x to buildpacks:y, to get a new feature in A, usually doesn't affect B, but in the specific case of Python, means the installed version might change and break a bunch of stuff.

5. Python 3 changes often being bundled with breaking API changes in app updates, making "upgrading to Python3" a uniquely involved challenge for each one.

If you deprioritize "fault" for a second, such that we're not deciding who's to blame here (whether it's the transition plan by the language devs, the library or app authors, the distribution handlers, or even end users like me), you have to reconcile the fact that this is a unique situation to python. I am not describing the woes of development in general. I am describing a very python-specific set of experiences for non-python users. I did not out of nowhere decide to get angry at a language I don't use. This is the result of a repeated set of circumstances that just aren't happening elsewhere.

As has been mentioned throughout this thread, it is further upsetting that a lot of this could have been avoided. The fact that there are things that could have been done differently to make the long tail effect less painful is fairly disappointing. At the end of the day, I don't need a list of solutions. I've implemented them all, don't worry. I've gone through and read the 500-thumbs-upped threads of people repeatedly running into this exact same problem, either found an easy workaround or taken the time to update the config files for a tool that rudely put itself at the front of my priorities list in order to fix the build, or found a way to remove the utility in question completely. This is not me coming on here trying to figure out how to move forward. This is me identifying a clear pattern, and sharing what I believe to be is a common conclusion: putting "python" in the "cons" column when considering a tool.

I come from the node world. There's plenty of people that do the same for node. Some of it I consider fair, others not. But I make no illusions about one fact: if we want to change that perception, it's up to us to earn that by making the experience better going forward, not by explaining to people why their experience actually isn't real. For example, if there is a simple solution, if you just were familiar with the ecosystem, then LOG IT WHEN THE ERROR HAPPENS. If the person has to go to stack overflow and see a thousand other people running into the same thing, then you've failed.

> However that's not what you're complaining about. You are complaining about is that you want others to support python2 forever, just as you may want browsers/sites to support `alert()`, the `<blink>` tag and IE 6.0 forever. However that's not how the world works.

This is also hilarious since you had to scrape the bottom of the barrel to find the only instances of "non backwards compatible" features in JS/HTML, an environment notorious for backwards compatibility. Also, `alert` may still stay in.

Python is basically in the same situation as Perl was a decade ago, with the added benefit of mainstream, living-breathing "success stories" like all the data science stuff (numpy, scikit-learn, etc), django, etc. (And Python seems to have a sufficiently large batch of pragmatic & progressive experiments that address various challenges facing users. Decades of effort went into scraping off the warts in areas like packaging, performance, stdlib ergonomics, etc. Python also successfully reformed its leadership.)

Avoid writing huge, long rambling comments and try to simplify your arguments to less than 100 words. You may find the fault, and the fix, becomes obvious.

Start with trying to refute "The python2 executable still functions correctly as does all other code you have that relies on python2" without referencing anything specific to your build practices.

> This is also hilarious since you had to scrape the bottom of the barrel to find the only instances of "non backwards compatible" features in JS/HTML, an environment notorious for backwards compatibility. Also, `alert` may still stay in.

Those where the first that came to mind. There are many others, and many more if you widen it to server-side JS.

If you don't want to read the comment, then also don't bother responding. The comment is long because I am explaining a death by a thousand paper cuts situation. Don't worry, Python isn't going to die just because I don't like it. You don't have to defend Python even if you don't feel like reading my comment. You can also just ignore it or come back to it when you have time to engage with it thoughtfully, instead of just asserting you are correct and having your only argument be that I can't fit my argument to the arbitrary word length you've come up with.

> Those where the first that came to mind. There are many others, and many more if you widen it to server-side JS.

You should stick to talking about Python. This is just not the case and not the culture of browser JS. It's honestly fairly disrespectful of the monumental work browser engineers have done to make that a reality. If you want to switch your argument to server-side JS (where blink and alert don't exist and you are thus leaving it completely as an exercise to the reader to come up with the similar situation), fine, but node 0.10 runs exactly the same today as it did 10 years ago. Again, the bar I am setting is low: have the unchanged thing still gettable and still runnable. I get it. When you work in a certain environment, you imagine all the problems it has must be common and all the benefits are unique. But that isn't the case with this particular problem.

Start with trying to refute "The python2 executable still functions correctly as does all other code you have that relies on python2" without referencing anything specific to your build practices.

Presumably you’re too “not caring about any of this” to just pin your image to “alpine:3.15” and have it just work exactly as before? And presumably you’re too not bothered with any of this to understand that you had to do this because you didn’t pin the image to begin with, so your builds are not reproducible?

And this is anyone else’s problem why?

My point has always been simple: Python causes more pain than other languages when you're not directly using it. I can get node 0.10 on alpine 3.17 just fine. This demonstrates that your comparisons to server JS are completely wrong. But I know, you're going to tell me this is my problem and I expect the world to do work for me or something. Nope. As I've repeatedly stated, I've taken the time to get it to work already. I know you want to believe I'm here waiting for you to fix it, but I'm not. All I've said is: "Huh, I don't have this problem with anything else. I'll keep that in mind before I decide to use stuff from this ecosystem in the future".

The argument I’m making is none of your issues stem from Python specifically. As others have said, you are conflating your own confusion and unwillingness to understand with systemic toolchain issues that do exist. Except, you are not being bitten by those - you’re still at the “shoot myself in the foot and blame the gun” stage.

What a slight of hand to say "You can get it the same way you can with node, except for you can't". You should go into politics. Anyways, you asked me for a simple example, I gave it, and now you want to say what I want doesn't count because "no one wants it". Just choose a different challenge next time.

> thus you’re complaining that nobody else is maintaining this port for you.

I'm just pointing out a discrepancy with other environments. One you initially denied existed, and once I trivially pointed out did exist, you switched to attacking the complaint itself. Look, if I'm looking at two toasters and one has less features, sure, you can bark at me and tell me to make my own toaster if I want that so bad. OK, sure. Or I'll just get the one that has the features I want. You are still allowed to believe that those features are useless, but you can't deny they exist. It is OK for you to disagree with the properties I value in an ecosystem, you don't have to resort to telling me I don't know what I'm doing just because I value a different set of properties in the ecosystem.

Someone has glued on a big red alarm clock onto the side of one of the toasters.

Your conclusion after seeing this is to write to the manufacturer of the toaster without the alarm clock glued onto it and complain that it’s lacking a big red alarm clock and say they suck because they are not copying the amazing alarm clock features that the other manufacturer has.

Ok.

Nobody is threatened by anything, and my point has nothing to do with Python.

If someone posts a comment saying “curl sucks! Every time I run it it just prints ‘command not found’” I’d of course try to tell them that’s not a fault with curl.

If they replied with “yes it is because I can’t figure out how to install it from this 3rd party site” then the discussion may become protected.

But ok, maybe we will agree to disagree.

Afterwards you’d get involved in a long weird analogy discussion about two people arguing about git. But that fault would be shared by both of them.

You won't know what version of Python 3 you're getting, but at this point it's safe to assume you're getting a version of Python 3 when you call python.

Only old scripts written when Python 2 and 3 were maintained separately should've ever had this problem, but we're past that point now.

How about one with dataclasses support? Again, you get to install a new interpreter everywhere.

Python packaging continues to be a sore spot for the language.

However, you can ensure compatibility around this by specifying the required version in the top of your script:

    import sys
    minimum_version = (3,6,1)
    if sys.version_info < minimum_version:
        sys.exit("You need Python 3.6.1")

As long as you put this above any code requiring any specific version (this will work with Pyton 2.7, I don't have anything older to test) this will halt execution on dependency failures.

You can include these checks in your setup.py if you want to rely on Pip so users are warned on install time. You could probably also add this to some wheel trigger if you add a local/public package as a dependency in your requirements.txt.

I mean the Python ecosystem is such that now I have to "package" up this script to potentially tell the user with Python 3.6 that they need to download a new interpreter version (eg 3.9) to run my application (and any dependencies). This puts all the pain of running a script on every user that wants to run it.

I've said it before, but Python requires users to replicate the developers environment...and somehow this is fine?!? Statically compiled languages (eg Go, Nim, Rust, C/C++, Obj-Pascal, D, etc) ship binaries...that just run.

Obligatory xkcd: https://xkcd.com/1987/

When in doubt, add the #!/bin/env python3.10 shebang to the top of your script to ensure the version it's running on is the same as the version you're writing the script for.

If I build an application on Linux I need to specify a glibc version or a PHP version or a Java version in some way as well. I don't see what this has to do with Python. Good luck running a jar built with JDK19 on JRE8, or an executable linked against glibc2.36 on Ubuntu 18, or a Python file written for Python 3.10 on a system that comes with Python 3.6.

Statically compiled languages come with their own problems (note that C++, Go and Rust also have system requirements, I've had to deal with a Go executable that would not work in a musl environment and a Rust executable that I needed to build on Ubuntu because the compiled version from my Manjaro laptop linked a against a libc version that was too recent).

If you need dependencies, specify and install them. If your only dependency is the interpreter, document it, specify it in the shebang, or deal with it. Python 3 is completely backwards compatible in my experience, it's only the programs that stuck with Python 2 for a decade that have trouble running on modern machines in my experience.

Yes, you are starting to see the problem. This is why I recommend statically compiled binaries as a solution to this scenario, and do not recommend any of the above.

Here's a real scenario I've had in my career, when deploying a script to 350k systems. I know I have Python 3.x, but I cannot count on the fact that every (or most) clients will have even a functioning Python interpreter. So what's the solution? Ship a docker container to everyone? There are multiple ways to solve it, but I complied a statically linked binary instead of writing a Python script...and had no issues.

> If you need dependencies, specify and install them.

Here be dragons, and this is the entire reason I avoid Python when I need my code to 100% work on another machine I can't fully control. I choose statically compiled languages in these instances. In a server deployment...sure it's fine because I can control that.

> Python 3 is completely backwards compatible in my experience

I do not agree. If you write a script that uses any of the following features and try to run on $version -1 (or more), your script will fail.

  * 3.6, F-strings were introduced  
  * 3.7, Dataclasses and introduced 
  * 3.8, Walrus operator (:=) 
  * 3.9, merge (|) and update (|=) operators added to the built-in dict class
  * 3.10, structural pattern matching
  * 3.11, exception Groups and except*¶ 

Maybe you'll say to just install a new version of Python. All that is overhead and requires more management. Ok, that's fair.

The problem is when you want to run that script on a bastion host, or a prod server, 350k machines, or a server you don't have ability to download and install external binaries? (all of these are real scenario's I've experienced).

My experience has taught me that if I can't control the machine the script runs on, I save headaches by using statically compiled languages. As always, YMMV.

Isn't this precisely the problem containers are trying to solve? You can update your system, get rid of unsupported software, and still keep the older version of Debian within your container that depends on an unsupported version of Python.

  FROM debian

  RUN do-x
  RUN install python2

Older versions of debian that still support python2 will theoritically be around forever, and any codebase that absolutely needs them should always work.

(though I contend that, given that we've had 15 YEARS of warning that this was coming, such instances should be vanishingly rare and not under active development)

This right here is the part that I'm still having troubles wrapping my brain around. People still stressing about Python 2 "going away" (it's gone folks; accept it) despite the fact that there's been well beyond a decade of advance warning, and Python 2 having been officially EOL ages ago now.

And than they will start complaining that it's other peoples fault that they didn't move.

Also, at some point old OS package repositories may be deleted (or bitrot out of neglect)- at that point building containers that depend on python2 will fail. You'll be able to use images built previously, but not everyone is backing up custom images in some remote docker registry.

The only reason I can come up with is that the installation process is wrongly changing the `Python` symlink. Anyone have other known causes? Does it screw with dependencies?

The worst part is a lot of times, since Python 3 is a breaking change, people decide that it's a great time to finally do all their breaking API changes too, so all of a sudden updating something to the Python 3 version means that all of its associated config files have changed format, and now you're updating config files that have worked great for ages and running into bugs there.

Your issue with Python is that some people having bad development practices?

However, I will take your snarky reply and turn it into lemonade by using it as an opportunity to talk about how often development teams fail to consider the social implications of their changes. As is mentioned in another comment on this post, the Python team seems to readily admit in retrospect that they didn't predict the pain of having no clear transition path, etc. I think another lesson to take from this experience is the realization that if you create some big "line in the sand" update, you may absolutely inadvertently encourage bad development practices. If the Python 2-to-3 transition had held "ease of transition" as a core value, then it is more likely that that value would have been modeled downstream. By instead initially declaring that the 2-to-3 transition was going to be some watershed moment and people were going to have to update everything, it is not surprising that it could encourage people to just pull the trigger on a bunch of breaking stuff they wanted to do, and come to the (incorrect, but understandable) conclusion that they didn't themselves need backwards compatibility since the entire ecosystem was going through a "unique" non-backwards-compatible change anyways.

Most of the major Python packages did maintain 2 and 3 compatibility at the same time. Maybe a few packages had bad coding practices but you’re going to hit that in any language.

I think you’re extrapolating unfairly

But that's exactly the nature of doing this sort of change. Again, I am not a Python person. This transition created an unintentional filter where the utilities with poor practices surfaced to end users, such that their only direct interactions with Python are always negative. So sure, the extrapolation may very well be unfair, but that is the completely avoidable situation they set themselves up for. No other language expects end-users of the programs written in that language to be sympathetic to some internal engineering transition. All this stuff may have great reasons, but you've already failed if you have to explain those reasons to non-Python users, or tell them this is confirmation bias or whatever. It's just not happening with other languages. And this is for big stuff too. Ansible, which uses 2 pythons, one locally and one remotely, was a pain. Add on to this that this was a fairly public boondoggle, that the makers agree wasn't handled well, so whether "fair" or not, this is a common take away I think.

Whether that’s libc requirements for compiled languages, or Perl language versions being breaking, or even as subtle as differences in behaviour between different compilers such that your system may give different results than another user.

So I disagree with your assertion here.

Yes it’s unfortunate that this is lifted up to the user, especially because of the fact it’s interpreted, but it’s not anywhere near unique to Python.

Therefore I think it’s not a logical outcome to outright avoid Python.

That does not dispute the original claim though. It's still a problem for Python even when other languages have the same problem.

> or Perl language versions being breaking

Perl 5.0 code written in 1999 will still happily run in 2023. In part due to the fact that breaking changes are *opt-in* within a Perl script.

Reference: https://www.perl.com/article/announcing-perl-7/

"""Perl 7.0 is going to be v5.32 but with different, saner, more modern defaults. [..] Perl 5 still has Perl 5’s extreme backward compatibility behavior, but Perl 7 gets modern practice with minimal historical baggage. """

Honestly, Perl is an example of extreme backwards compatibility that has the user experience at the forefront of their design decisions.

I’m saying that avoiding Python because of it is illogical (to me) because the problems they’re describing aren’t unique to Python, and I don’t think there’s much that doesn’t suffer from it.

Perl is an odd one because, as your comment says, Perl is going 5->7 because 6 was such a disastrous break.

ack

> I’m saying that avoiding Python because of it is illogical (to me) because the problems they’re describing aren’t unique to Python, and I don’t think there’s much that doesn’t suffer from it.

I don't completely agree, but I see your point. Any interpreted language (or non-statically linked binaries) could (and does) suffer from these problems. I'd like to see Python have a better packaging story...but for now we live with shipping the developers environment (via docker or virtualenv) for running complex Python applications and services.

I will say that *statically linked* compiled languages are able to avoid many of these problems because running these binaries typically only requires the OS (eg, Go, Nim, Rust, Zig, C/C++ when statically linked, Object-Pascal, etc)

> Perl is going 5->7 because 6 was such a disastrous break.

Agree. And they did the right thing by actually renaming it Raku and skipping the version release.

It was to precisely avoid this scenario where you get it switched out from under you.

This doesn’t mean every distro followed suite (though most popular ones were good about it) , but the biggest issue is users. Tons of users create the symlink or alias for convenience and then forget about it till stuff breaks.

> Distributors may choose to set the behavior of the python

> Avoiding breakage of such third party scripts was the key reason this PEP *used to* recommend that python continue to refer to python2.

Emphasis mine.

Same Issue with e.g. converting Vue 2 to Vue 3 or Angular.js (Version 1) and Angular (Version 2).

I really hate when some programming languages or frameworks completely remove important functions which were used by nearly every project.

It's kind of a shame how this transition took more than 10 years. The good part, we learned that we don't have to go through the same again. of course I dropped python (I was like 18 years old and didn't know anything about programming), I had to learn python last year and you still can see a lot of stuff that makes the experience kind of annoying.

Anyway, I'm pretty impressed how Python still is in the first positions of most charts, I still fail to see how great the language is, but hopefully I will be able to someday soon :)

I fail to see how that is true. Python was already catching on in a big way back during python 2.7 and I'd guess it probably lost at least 18 month worth of momentum building with python 2->3.

I'd say Pythons popularity today is at best unrelated to python 3.

languages that are actually growing in userbase/popularity are not static. this is hard to accept for many users of popular languages.

Huh? This has a strong "No, it's the children who are wrong" vibe.

But to be more fair to your point, I think this is a dismissal of the OP concern's. A troubled ecosystem should be concerning to the Python community as a whole because it threatens future adoption if not addressed.

User's don't want a troubled ecosystem and will find languages that avoid these problems...and Go, Nim, and to smaller extent Zig, Rust, D are strong alternatives for the different workloads currently provided by Python tooling.

Python has a good moat with both ML and mind-share of developers...but at one time Perl had a similar moat...

> User's don't want a troubled ecosystem and will find languages that avoid these problems

It's true, I'm very late to Python myself -- for the exact same reason. The distribution part of the ecosystem is a nightmare.

People want utilities, but they learn more than they ever cared to because of dependencies.

I'm only learning how to manage it now because work demands it.

Prior, all of my needs had fortunately been met by my Linux distribution of choice -- the Python part was abstracted away, for the most part.

Developers of these utilities could save some (shared) effort by offering package specs/finding a maintainer, but I understand why it's not common.

(I may even build/maintain it for you, to sweeten the pot)

I guess the fuel of this thing is, 'misery loves company' -- I'm dealing with it, and I can't just whine about it.

The "I don't use it and it still affects me part" was the bait -- they do use it

Not sure where you're going with this. I didn't whine to someone else to fix it. I did take ownership. Over and over. I fixed these problems, as unrelated to my main goal as they were. I then complained about it after the fact. Seems like a totally fair ordering of events to me.

Separately, I will state that Python is an environment that seems to go out of its way to make it hard to meticulously manage dependencies if you want to. There's all sorts of competing environments and package managers. I'm sure it's straightforward if that's your main environment, but it doesn't help if you're coming form the outside.

> Those should be routinely addressed as part of your auditing, along with the whole of the supply chain

This assumes that the only available option is "manage this dependency better". That's not the only option. There is also "consider using something different that is easier to manage." Most of these utilities exist in a competitive market. And if there are utilities that allow me to choose the cadence and separately make that auditing simpler, then I will choose that. That is the point of my statement. I am not choosing to go on a hunger strike or avoiding some no-other-choice tool. Quite the contrary, a lot of these tools were chosen by other members of the team at some point, either because it's what they were familiar with or some other unimportant reason, and now they have, to their own detriment, made themselves the squeaky wheel, and opened up the range of possibilities to everything from "put in the work to maintain this tool that has currently demonstrated itself to be brittle" to "maybe I should use something else". Generally speaking, it's a bad thing to only draw attention to yourself when you are doing something bad.

> can't really prescribe much... this is just generic advice.

And yet this is a specific problem.

> I understand the woes, it's just not as unique as this is painted

It very much is unique. I'm literally not experiencing this with any other language that I don't directly use. Do I experience this with the libraries I directly use in my language? Of course. I signed up for that. I know the risks I take using beta builds and have a keen sense of the current internal transitions, etc. But I absolutely do not become aware of the latest argument in the C++ language spec, or a contentious Swift proposal, or end up becoming vaguely aware of the string implementation choices of a language I don't use. That is a unique Python thing. The fact of the matter is, Python has (recently? momentarily perhaps?) proven itself to require more maintenance than other tools I use. The community can decide who's at fault for this (language devs, library devs, end users, whatever), and even whether or not it matters, but if the message to the people outside the community is to deny that this is unique or troubling, then don't be surprised if the way they choose to take ownership is to count Python as a point against a project they are considering to use.

It was very knee-jerk to the interpretation that I had: "distance has been kept from Python and yet it's been a problem"

There hasn't been that much distance, truth be told. You did the same care for Python as you did the libraries, but it cost too much.

We're in complete agreement on dependency/manager aspects of Python -- my main point is, that's the core of it, I guess.

Everyone goes through this, it's awful and annoying, and my original post is a poor response to the presentation of that

The Python 2 vs 3 thing is settled, upgrade to a recent LTS and it's done... until a random relic utility comes around. Then obviously discard it

OMG, that's a parody I hope!

Congratulations to Debian on upgrading to Python 3!

And what surprised me most is that every time I mentioned this, there would be lots people telling me how this is a superior design because you can operate string like bytes. I mean, it of course has its upside, but I don't think it's worth it if you care even slightly beyond ASCII.

For me, the unicode handling alone was worth the time spend migrating from Python 2. That was a decade ago, to finding that the "python" command still launches a Python 2.7 interpreter in 2023 is just beyond belief. Personally I feel like they should have yanked Python 2 in Jessie (Debian 8) in 2015, more realistically in Stretch in 2017.

I'm sure you could do things well in Python 2 with proper combinations of encode/decode, but it wasn't obvious where you even needed those because with ascii text, things "just worked" transparently. With Python 3 it's very obvious where you need encoding/decoding because bytes != str.

In the early 2000s, I maintained an py2, wxPython app with the users having the system encoding win-1250; the effort to patch this was unbelievable. The migration to python3-style handling forced everyone to think about these issues, not just few people for which things were crashing. Even just popularizing the issue was great, until then, many maintainers of third-party libraries didn't even understand what is the problem that you want to "needlessly complicated" fix in their libs.

The problem is not the end-user invoking the command.

The problem is scrips expecting `#!/usr/bin/env python` to invoke python-2.

apt install python-is-python3

For example GnuRadio started supporting Python 3 with GnuRadio 3.8 released in 2019, and then you had to port all your programs using it to this version. So no, in 2017, the ecosystem was not ready.

In order to support Windows better, Python 3 introduced support for UCS-4 (or worse, UTF-16) strings (depending on a compilation setting when Python was compiled) and they had to introduce extra string types to distinguish readable strings from binary strings ("bytes").

These extra types made Python 3 a lot harder to teach (I teach 30 person classes every year).

So it's not all roses now.

In the end, I got used to it, BUT I just gave up asking encode()/decode() questions at the exams. Very few people understand it, or care enough (and I understand why--it's a ridiculous thing to have). You only need it if your OS somehow slept through the introduction of UTF-8, which is backward compatible with ASCII, resilient even if there are transfer errors and can encode all unicode characters.

Encoding problem used to be really common in UNIX (and before that, in mainframes), but with the introduction of UTF-8, all encoding problems I had vanished and never appeared again.

Even Windows 10 has an UTF-8 mode now and the Windows API functions that end in "A" can be made to use UTF-8.

Now, in a sense, Python 3 has this entire complication for no reason.

That said, Python 3 is ok to use now--and, conceptually, distinguishing byte strings from unicode strings is better (for example so that you don't accidentially print the former to the terminal). It just uses up brain cycles that you could be using for solving your actual problems.

I get it from the "pass the exam" perspective, since that's one more thing to worry about.

But from my experience in teaching others, doing the conversion between bytes and string implicitly (à la Python 2's way) hinders actual understanding of this very important concept, and it's quite harmful in further study.

Bytes should be considered as a separate, more low-evel thing, away from int/float/strings; at the very least, it should be considered as bits/hex numbers. If you want strings, you explicitly encode/decode them in a way, even if everything is UTF-8.

On top of that, "byte string" is just a confusing concept. It might works for English speaker (by "it's a ridiculous thing to have" I assume you mean that, "'english'.encode() is just b'english', why bother?"), not at all for Chinese speakers, even in UTF-8. There is no b'中文' -- only b'\xe4\xb8\xad\xe6\x96\x87' which has zero meanings in their own.

And even from an easy-to-use perspective: most people don't even work on bytes often nowadays. A more abstract "string" type is all they need, without worrying about how it works under the hood (and if they do, they need to understand how encode/decode works properly anyway).

There was no conversion. `bytes` and `str` were the same type.

http://docs.python.org/whatsnew/2.6.html#pep-3112-byte-liter... says:

> Python 2.6 adds bytes as a synonym for the str type, and it also supports the b'' notation.

I just checked in Python 2.7:

    >>> bytes is str
    True
    >>> print("Hänsel")
    Hänsel
    >>> "Hänsel"
    'H\xc3\xa4nsel'

The point is when bytes is str, everything works just fine in Python 2 Linux with UTF-8 locale (which are used in all modern Linux distributions). No need to have a distinction between bytes and str.

That how the rest of the OS works, too. Even a lot of Gtk, Glib and so on (for example the GNOME desktop environment) assume that you are in an UTF-8 locale for file names, for example.

> A more abstract "string" type is all they need, without worrying about how it works under the hood (and if they do, they need to understand how encode/decode works properly anyway).

Ehh, we had students write drivers for measurement apparatuses and they all used Python 2 str (without being prompted to do so). No encode or decode anywhere. Of the students, almost no one who tried Python 3 for that stayed with it (instead they were using Python 2). There was just no upside for this use case.

I agree that, long term, having a distinction str vs bytes makes sense. But then you ARE juggling things that the OS doesn't need--it's basically busywork in Linux.

I'm not trying to minimize your experience--but I don't think it would happen if you tried python2 on Linux today. Not sure it was worth it breaking compat for that.

That's not true at all. I remember all kinds of encoding errors when dealing with the FS, the network or any user input when using Linux.

Unless you're talking specifically about IDLE?

Debian had Py3 for ages. It wasn't upgrade, that happened ages ago, just removing old packages kept for compatibility

The print-as-a-statement was ugly but convenient and didn't seem like a big deal, the integer division was something that you could live with once you knew about it (and you still need to know what the current behavior is), and the new string types make code more verbose, and the way they were handled introduced countless hard to find bugs during the migration.

Core developer Brett Cannon's talk "Python 3.3: Trust Me, It's Better than 2.7" goes over most of the differences at the time (2013):

https://www.youtube.com/watch?v=f_6vDi7ywuA

The fact that such a talk exists shows that the improvements aren't worth the switch for most users. If they were, they wouldn't need convincing.

Yes, Python 3 is (mostly) better than Python 2. No, it's not even remotely worth the amount of confusion and work it caused. If Python 3 had brought massive performance improvements, or proper support for multiple threads, then maybe it would have been. As it stands, Python's biggest weaknesses remain unaddressed.

Have you worked with pervasively non-ASCII text in Python 2? Like on a machine where any file might suddenly turn out to be non-ASCII (even if it’s only in a comment), not just data inside a few carefully-patrolled fences? Outside of a few well-behaved libraries (Flask), my experience was that it was utterly impossible. More than half of my time debugging straightforward code was spent chasing down and fixing UnicodeErrors, knowing that some will still remain. And that includes code that used nothing but the standard library.

Now you might think that’s not important, and though I disagree (especially when talking about “most users”) that would be fair. But it is a substantial improvement going from 2 to 3.

There’s no need to sugar coat it.

It was incompetently managed, in a way that made the technical success of the work look like a failure.

That’s on them. They screwed up.

I lived through it; I feel no particular need to smile and nod and say “it wasn’t that bad”. It was bad.

Things are good now! Python 3 is great, it’s well supported and the people involved all learnt a lot about how important managing communities is, as well as writing good code.

So, all round, a success!

…but, if you don’t acknowledge failures in the past, you’re doomed to repeat them.

> Now you might think that’s not important

I don’t think your experience with Python 2 is not important; I had similar issues with it.

No one is seriously going back to Python 2 at this point.

However, the process is a lesson worth studying; and the take away is not “what a technical success!”

… it is: never do this.

Big scary breaking migrations are bad, they hurt your community, and leave bitterness behind them.

Even if the result is better, technically; there are other factors that need to be considered too.

No, the community is stupid and wrong. My machine learning professor (actual professor, not a teaching assistant) only used Python 2 in 2017. Imagine how many people had their opinions shaped by someone like them.

> … it is: never do this.

By that standard, we would never get things like angular 2 or Raku.

Would it be better if we renamed Python 3 to some other name and pretended it was a completely new language?

Quite possibly yes? It doesn't run python2 code, it can't be inter-linked with python2 code, so is it really the same language? The hybrid dialect that works in both does exist but you have to have compatibility shims in a few places.

By comparison, most C compilers will let you use C89 and C17 code in the same project provided you set the compiler options, and C# provided the "netstandard" target for libraries to smooth the transition between Framework and Core runtimes.

Sometimes.

I mean, Russ basically said that about all the work people were doing on the golang package manager and did his own thing, and it worked out. Good job! Sometimes, a smart person can do something that is better when they're deeply steeped in the domain.

> No, the community is stupid.

I personally find that most communities have a lot of very clever, very thoughtful people in them, and if you listen, you can learn a lot from them.

Just because they don't agree with you, doesn't make them stupid.

I think that its basic respect to listen to your community, acknowledge them and think about what they say, even if you go your own way.

...but, if you (as a maintainer) just think they (your community of users) are just a bunch of idiots, do whatever you like. They are idiots if they hang around in a community like that.

However, I am firmly in the Python 3 camp. It is a better language (all the newfangled bullshit notwithstanding).

The print() function was absolutely not worth all the carnage it caused. They could have just kept the print statement around and introduced a printing function with a different name, for example.

The fact that they removed syntax for Python 3 meant that all kinds of scientific computing code had to be transitioned even if it didn't use any strings!

The Unicode handling on the other hand took ages because string handling is not easily searchable and replacing it requires understanding what the piece of code actually does.

So you have it exactly backwards: if the only migration was the print function then it would be a quick and relatively painless process. But fixing the hard thing which was the broken str/byte model was the hard transition.

I am curious if you actually went through this transition or if your opinion on this is more based on observation of others’ work.

Of course not, you’re just going to revert back to the version of bash that doesn’t require parentheses and sit on it for as long as you possibly can.

Code exists in three states: actively maintained, done/mature, and obsolete.

Python 3 attacked mature code, forcing developers to either go back into active maintenance or to obsolete it early.

I did transition some Python 2 code to 3. But mostly it was uneconomical and risky, so the vast majority of it sat on Python 2.7 until it was no longer needed.

I feel that most of the people who had a positive experience with Python 3 were people doing webdev using Django. They were working on monolithic actively maintained projects and so transitioning to v3 was just a normal maintenance activity.

Again, you are arguing that running 2to3 and testing your code was hard. I guess that may be true. But having hunted for a ton of subtle Unicode bugs that problem was very worth fixing.

It's also not a problem if you only have ASCII strings.

Didn't the unicode changes go on until at least Python 3.6? The issue that most C APIs did not care about their byte strings having any encoding at all seemed to cause one headache after another, especially on Posix, where everything touching the operating system could return random bytes. Reducing the scope to unicode wouldn't have sped up the transition one iota.

For comments and string literals in Python code, note that Python 3 changed the default from Latin-1 to UTF-8, but you can still use -*- coding: -*- to change that. Bytes literals outside of ASCII range you’ll still have to escape. As for program input that’s possibly invalid UTF-8 but not in places you care about, you’ll need to set errors='surrogateescape' or similar explicitly (or possibly set an 8-bit encoding like Latin-1, though not if there’s a chance of UTF-8 in places you do care about).

Paths are a bit painful, yes, both because there’s nothing on Unix systems that precludes /home/alice having filenames in UTF-8 and /home/boris in KOI8-R, and because NT paths are not byte sequences at all (they are WCHAR, that is 16-byte-number, sequences, in no way required to be representable in the current CP_ANSI). Having uninterpreted byte sequences as filenames would work to solve the former issue, and it does: I think most functions in os will accept bytes paths as input and treat that as a signal that you want bytes paths as output (if any). For the latter the cleanest solution is probably an abstract Path type—and you get one, in the standard library’s pathlib. That didn’t work all that well in early Python 3 versions, but it does now. (And IIUC inspired Rust’s abstract-type solution to the problem.)

Python 3 takes this fact and runs with the stance that strings should, by default, carry their encoding with them (and that encoding is specifically, more or less, a sequence of quasi-Unicode code points with unspecified internal representation). But this is an intensely half-baked solution to the wrong problem, because you need to know much more about a string than its encoding for just about any string operation you'd want to do. Just with Unicode code point concatenation, you can run into plenty of trouble with: - strings containing control characters - strings with unpaired surrogates (which okay, shouldn't really be there, but get used for various reasons anyway) - strings containing combining characters - strings in different natural languages - strings in languages with ligatures - strings with ligature-like emoji (and this is obviously not remotely close to exhaustive.)

So personally I'm ok with Python 3 but I actually never ran into the described problems despite using Python 2 for years, so it was at least not a universal problem that needed to be solved in such a breaking way.

I didn't insinuate that there weren't problems or that I wanted to keep Python 2 - just that it was perfectly normal to use it and not run into problems. I think I only remember a handful of encoding problems with my other python 2 adventures...

That's reading far too much into a tongue in cheek presentation title.

Besides, good devs are typically reasonably skeptical of new and shiny.

And when you've got a large Python 2.7 codebase, and Python 3 is backwards incompatible, you'd be looking at a lot of work to upgrade, so damn straight you'd need convincing to make the investment.

Hell, look at how many Java codebases are still 1.8 because of the upgrade cost of going past that, even though modern Java has many compelling features.

And expecting Python 3 to remove the GIL is incredibly unrealistic. How much Python code in the wild implicitly depends on the behaviour of the GIL? All of it.

About as much as depended on Python 2-specific features?

Removing the GIL would be bloody fantastic, but it's far more involved than 2 to 3.

And, the latest iterations of Python3 have real usability improvements - the error message improvements alone make development a lot nicer. And the perf improvements and jit coming I think in 3.13 will really make more people consider py3 for their new projects.

More like it shows that many people like GP are only aware of the superficial changes.

Also, the fact that there is a talk with such a name does not mean anything other than that someone is out there trying to clarify what the transition is about.

(There could have been a video "Windows 7: Trust me, it's better than Windows XP" and you probably still would say that it showed that improvements aren't worth the switch for most users.)

The first python 3 versions were not great and were missing some things. I think 3.3 didn't even had pip builtin

I ask, because I know I would have wanted that.

    use v5.24

The Py3 approach was terrible and wasted untold amount of hours just because you had to migrate everything, you couldn't just upgrade codebase piece by piece like in case of Perl.

At the very least they should've just made new one be under .py3 and just make Python3 interpreter transpile Py2 code at runtime

.. but you can't use old libraries in new code.

I think having Python 2 vs. 3 is a clearer demarcation, myself, but the Perl community cares a LOT about backwards compatibility.

The Python mess was, IMO, a typical example of bureaucracy inventing for itself new but previously unnecessary work to justify its existence, so I agree that that the decisions of how to introduce Py3 features caused (and still cause) waste of immense amount of the hours world-wide that could have been used more productively. Sad.

There was some breakage between 5.6, 5.8, and 5.10; Unicode is hard, and it takes time to get it right. But I think the key difference I've heard as a mostly python avoider is the intent was for code written for 5.6 to probably work in 5.8 and 5.10 and if it doesn't, for there to be a way to have one file that works for all versions.

From what I understand, it's not easy to have a python module/script that works in 2 and 3, and you can't go to 3 unless your dependencies do, so if you have a lot of dependencies (as is modern), you're stuck on 2. Your dependencies won't want to move to 3 either, because their users are stuck on 2, so if they just switch to 3, they're droping users; instead they need to support two parallel versions of their code. Most perl modules didn't have to do anything special to support 5.6 and 5.8, but if they did, it was usually small and it could be done within the file with conditional compilation --- I don't think that was an option for python.

I think very few people had trouble remembering them, and if you're using one, you'll be looking up documentation for anything with more than a few methods.

It really is frustrating in these situations with high level languages. It's super possible to do the majority of changes they did in non breaking ways.

Hell, they do it already with the __future__ package. Where did all that compatibility work go in python3?

Also, Perl 5 is a very different example. While I haven't followed Perl for quite a while, ISTR that Perl had a very similar transition from 4->5 (I personally remember spending a lot of time trying to get old code working on newer OSes). Also, AFAIK but I may be wrong, Perl 6 has not really taken off like Python 3 did.

That's terrible practice.

If you don't write special in your code, Perl doesn't have a "say" method (because it didn't have one in Perl 5.8). If you declare in the code that you're code for Perl 5.10, magically there are all the Perl 5.10 features, including "say". You can also declare you want specific features, rather than declare a specific Perl version.

  $ perl -v | head -2 | tail -1
  This is perl 5, version 30, subversion 3 (v5.30.3) built for x86_64-cygwin-threads-multi
  $ perl -e 'say 1'
  Number found where operator expected at -e line 1, near "say 1"
          (Do you need to predeclare say?)
  syntax error at -e line 1, near "say 1"
  Execution of -e aborted due to compilation errors.
  $ perl -e 'use 5.010; say 1'
  1
  $ perl -e 'use feature "say"; say 1'
  1

If only Python were as reliable for backwards compatibility.

The Perl community is going through their own Py2->3 moment, and they're doing it with great care. They're going to create Perl 7, which will be Perl 5 where the "defaults" make the language look more like Perl 5.32 rather than Perl 5.8 [1]. Modern Perl scripts don't need so much boilerplate to enable modern features. They know exactly the trouble this will cause; if Perl 7 ever becomes the default Perl interpreter, the only one, it _will not run_ lots of very old Perl scripts untouched, because the defaults will make years worth of language changes suddenly visible to those scripts. Even if the fix is just to slap "use 5.008" on every script, they worry about it. That's a much better attitude than the Python developers.

[1] https://www.perl.com/article/announcing-perl-7/

> That's a much better attitude than the Python developers.

You do realize that this comes at a cost, right?

A lot of decisions in Python3 (and further) are based on the desire to keep the interpreter simple.

Python Core Team doesn't have unlimited resources, and even if they did, the cost of complexity grows non-linearly.

py2 -> py3 transition did not go as planned, but that doesn't mean that following Perl route would have had better outcomes.

Personally I only use python for throwaway scripts. Whatever is written in Python likely won’t work in 10 years anymore. So they’re all lost in time. Whereas I can continue some of my C++ eternity projects and they still work perfectly fine.

Certainly today DNS domain names, URLs, and email addresses can – and do – contain >0xff input, and for some of these that was the case in 2008 as well (URLs and email addresses – IDN didn't come until a bit later).

The Python 2 situation was untenable and lead to a great many bugs; "decode errors" were something I regularly encountered in Python programs as a user. In hindsight, the migration effort for existing codebases was understated and things could have been done better with greater attention to compatibility, but the problem it addressed was very real.

Is that true? My understanding is that DNS never supported non-ASCII and so punycode was invented.

I can count on my fingers the number of places I invoked manual DNS lookups inside production code.

https://www.w3.org/International/articles/idn-and-iri/

https://www.rfc-editor.org/rfc/rfc6531

etc...

Just because you can get away with ignoring the non english speaking world doesn't mean Python should pretend it can.

As for RFC6531, my understanding is that virtually no email provider implements it, because of the same risks that make browsers often show IDNs as their punycode version - Unicode is extremely easy to use to confuse people maliciously or accidentally, since it contains vast amounts of duplicate characters (e.g. Latin a and Cyrillic а), even larger amounts of similar looking characters (e.g. Latin r and Cyrillic г), and even characters that are ambiguous unless you choose a locale! (the CJK problem, where the same Unicode codepoint can represent different characters based on the locale - whose locale to use when communicating between two machines being the implementer's problem).

Also, I'm not saying that Python shouldn't have had proper suport for separating byte arrays from encoded strings. I was only pointing out that there were actual legitimate use cases where a valid Python2 program was broken by the Python 3 Unicode string transition, whereas the GP was claiming the Python 2 had to have been buggy already.

Edit:reading around more, it seems that RFC6531 is getting some traction, and many providers accept sending to/receiving from internationalized emails even if they don't themselves allow you to have a Unicode email (e.g. you can't have айа@gmail.com, but you can correspond with someone having such an email at a different provider). So, email was a bad example in my list. The rest still stand.

For example, what if I want to enter "ουτοπία.δπθ.gr" in an application, via direct user input, a config file, or something else? Or display that to the user from punycode? No one expects users to convert that manually to "xn--kxae4bafwg.xn--pxaix.gr", and no one will understand that when being displayed, so any generic IDN/domain name library will have to deal with non-ASCII character sets.

The same holds true for email addresses: "ουτοπία <a@example.com>" is an email address. Sure, this may get encoded as "=?UTF-8?q?…?=" in the email header (not always, "bare" UTF-8 is becoming more common) but you still want to display that, accept it as user input, etc. People sometimes to forget that the name part of an email address is widely used and that any serious email system will have to deal with it, and non-ASCII input has been common there for a long time.

In specific applications you can often "get away" by ignoring non-ASCII input because you sometimes don't need it. For example I'm working on some domain name code right now which can because everything is guaranteed to be ASCII or "punycode ASCII", so it's all good. But in generic libraries – such as those in Python's stdlib – that's much harder.

How would you send an email to the customer support of flüge.de? How would you parse that domain?

IDN is here, and it's here to stay.

At least python3 got that right, unicode by default.

I don't understand why there apparently wasn't a clearly stated preference about `python` being Python 2 and `python3` being Python 3 from the Python org.

Hell, if you're still using 3.6 you're limited in your ability to upgrade dependencies, 3.8 is generally the minimum supported Python version these days.

It may have lost a lot of people who got frustrated by the 2->3 transition, but Python is clearly not in a bad place as a language today.

Where would they go from python? Node?

I tried using the library less than a year ago. I must have missed something. This would be great news… if the way you told me didn’t make me feel like shit.

Most actively maintained Python code works with Python 3 now, but there's still a whole lot of even maintained code which uses `#!/usr/bin/env python`.

Are there some valid reasons that made 2->3 migration insanely hard for some projects? I remember seeing blog articles whining about print vs print(), but surely there are some more important stuff.

A big part of 3 was Unicode strings. In 2 you could mark a string as Unicode with a u””. This would have been a great way to let libraries and code work with both 2 and 3. They banned this, but you had to still use b””. This was the attitude and just one example.

The list of stuff that got hard / slow was long. Migration path was unnecessarily difficult as was compatible code. Somewhere around 3.4 it was like a light switch flipped and they started being more reasonable. ASCII handling improved (yes, not all internet protocol stuff is Unicode), they began making it easier to target 2/3 etc. but it was horrible to start. I’ve thankfully forgotten some of the details :)

I feel this take is outright wrong. Python provided it's 2to3 tool[1] which took care of the bulk of the work required to port Python2 code to Python3. The only code that was not supported was eggregious errors in the code that worked by coincidence, such as handling bytes as strings and vice-versa. Porting old code to Python3 is a breeze that consisted of running 2to3, run tests with python3, and if anything breaked then just touch up the code to get it to work. I know it because I personally ported half a dozen projects throughout the years.

I'll go as far as to claim that most problems porting old python code to 3 were either upstream dependencies dragging their feet or internal human/organizational issues.

[1] https://docs.python.org/3/library/2to3.html

Your comment is a perfect illustration of the issues. Lots of user blaming. No actual solution. As I said, it did start to get massively better at some point. Instead of condescending lectures on org issues they for example began allowing u”” in 3, which did not mess up Unicode handling in 3

I don't see where GP made a claim that it would. Why would one expect code written for a newer version, using features that don't exist in an older version, to run under the older version?

Folks using u”” were already being careful with Unicode . Supposedly Unicode was important enough for 3 to default to it.

The cost benefit of then destroying code using u”” made no sense to me. It was unnecessary. This was followed by the oh just upgrade thing. I’d love to see the “quick” upgrade guidos employer did. My guess is a total lie that it was easy and it probably took years and VC type money.

It’s not about using new features, it’s about not unnecessarily breaking old or allowing for cross compatibility.

Except they broke the old way for existing features, required you to use the new way and in most cases you could manually write code so it would still work with both. Meanwhile the only tooling provided by the python 3 crowd screwed over anyone with an existing customer base stuck on python 2, which at least early on should have been a foreseeable problem. Not to forget that anyone stuck on python 2 was literally Satan and projects where outright "shamed" to drop python 2 support.

Many folks would have been ok updating to 3 compatible approaches if it didn’t blow up their 2 story. That’s what ultimately happened in lots of cases when it became more doable. The rip and replace everything at once was a weird goal

That's what "breaking changes" and "backwards incompatible" means.

What exactly is hard to grasp?

> So you ended up w a chicken and egg problem.

You really don't. Your code and your upstream dependencies need to be ported to python3. Once your dependencies are updated, all that's missing is you doing your job.

Python2 has been on the path to deprecation for how long? A decade?

You're fabricating problems where there are none.

> Your comment is a perfect illustration of the issues. Lots of user blaming.

There is no issue. When I had to port projects to Python3, I just ported them. No drama, no hangup. You're pretending there were problems where there were none, and you're throwing a tantrum when this fact is pointed out to you.

There is really no justification for this. It's high time people like you stop making up excuses and start to own up their misconceptions, misjudgements, and mistakes.

Some projects were rewritten from scratch in entirely different tech stacks in less time the likes of you complained they could not update their projects from python2 to python3.

What is it about python 3 that brings out this lecture stuff.

I would really like to see the inside of this “easy” change at even the bigger most well resourced places. Google / Facebook / Dropbox etc.

At small places where a program is working and programmer is gone they are not investing in ports.

Anyways - they did eventually get a clue. It’s much easier NOW to handle this, but it wasn’t at the start. Yes, they did put u”” back in. No it did not destroy the world.

This saga isn't completely over, over here: We still have code running python2 that we don't have the room to upgrade, and libraries that have to work with both python2 and python3. Meanwhile, people are pushing "code consistency" to the point where people outside our team are taking those libraries and running the "black" formatter on them - but "black" doesn't understand python2, so it removes the "u" prefix and breaks the library.

I think some people convinced themselves that using "" without the u"" was disenfranchising to people who don't want to use ASCII to represent their language. This transformed a practical argument over syntax into a moral crusade.

This isn't strawmanning - this is literally the case for a codebase I'm working with today.

Though I find it hard to believe that there are some useful dependencies that have not been ported and don't have better alternatives. Typically a "never ported" dependency is never ported because it is deprecated in favor of something better.

If you're consuming dependencies which were never updated in the past decade then you have more worrying problems to deal with than porting your code to python3.

Relax things a tiny bit in 3, back port to 2 via futures, get a library like six going. It could have been a ton easier.

In fairness by 3.5 maybe they’d realized this, but a lot of lecturing was done before. Initially recommendation was to upgrade all dependencies and programs to 3 and not try and make dependencies work w both

The other side of that coin is that the python maintainers went out of their way to push back the Python 2 EOL. They kept on pushing the date back and back again, as per the sunset page:

"We did not want to hurt the people using Python 2. So, in 2008, we announced that we would sunset Python 2 in 2015, and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020."

The drawn out demise of Python 2 was, frankly, painful.

I have no time for whinging snowflakes complaining 12 years (2020-2008) was not enough time to migrate their code to Python 3. Hell, even the original 7 years (2015-2008) should have been long enough for 99.999999% of the community.

I'm old enough to have read this when it came out...and it changed my view on backwards compatability (from Joel Spolsky of Trello, FogBugz, and StackOverflow fame)

"Code doesn't rust": https://www.joelonsoftware.com/2000/04/06/things-you-should-...

If a company has production Python2 application / service (with hundreds of thousands of LOC), what business value does it bring to migrate it to Python3?

At that point, if you've got to make severe changes, folks might decide to use a language that doesn't impose breaking changes (and business cost) on them. YMMV.

Not just talking ASCII was one of the first ways.

Drift in 3rd party library support is another.

Security support of the language and libraries is a massive one.

Simply put as hard as you try to stand still the rest of the world is not going to.

They had to push back the EOL because they screwed up.

I feel like a lot of the people who had this reaction were reacting to being told (correctly) that how they handled strings was broken. A big reason that projects had difficulty upgrading, especially difficulty using the automatic 2->3 upgrade tooling, was because of broken handling of unicode and broken handling of binary strings. I remember reading a quick assessment of a large open-source project that there were thousands of calls to a single string-handling function that would each need to be inspected by hand to determine whether it was correct usage that would be correctly translated by the 2->3 converter or buggy usage that needed to be fixed before running the 2->3 converter, with some people guessing a 50/50 split. The effort was going to be gargantuan, and though the ultimate cause and responsibility was poor design choices in Python 2, the immediate cause was that the code was wrong, and some people didn't appreciate hearing that.

No one really blamed programmers for getting strings wrong in Python 2. It was unreasonably hard and extremely rare for people to get it right, so there was not really any shame in Python 2 programs and libraries being pervasively broken in that way. But some people still felt it was smug and condescending to make reference to this elephant in the room. This put the Python maintainers in a bad position, because they were trying to explain the benefits of upgrading and also the costs and difficulties, both of which were intimately tied to the brokenness of current codebases.

As a user and spectator to the whole drama, therefore, I felt that some maintainers got overly defensive about issues that virtually every codebase suffered from. And some of them took a "so what" attitude towards bugs involving non-Western languages, saying that if they personally didn't care whether their code handled Chinese or Bengali filenames then it wasn't a bug to do random things with them, which sounded just as bad ten years ago as it does now.

Can you be very precise about what you mean by "broken" or "incorrect" code?

Python3 enforces the difference between encoded and unencoded strings. It forces you to deal correctly with unicode. If your code base was already handling unicode correctly it wasn't much hassle to migrate. If your code base was making a mess of unicode handling --like many at the time were-- you'd run into that headfirst.

I'm glad that this is what we have for Python now, despite the pain it caused. We'll be using Python for another few decades probably, so we'll reap the benefits.

edit: This talk he did from pycascades 2018 in particular I remember seeing and being a good retrospective: https://www.youtube.com/watch?v=Oiw23yfqQy8