[ wasn't an operator or language construct but an external program similar or the same as test. ] was a final argument.
In today's world where bash is and does supplant POSIX for all intents and purposes, use [[ and (( because they're part of the interpreter and more flexible.
When I need to write shell scripts, it's because I need portability, so I write POSIX shell scripts. If I can compromise on portability anyway, I jump right past Bash to a more sane general purpose language, like Python.
Until you realize that some of the expressions in [[ are evaluated as arithmetic expression which has some surprises around array subscript and command substitution.
In short, because the following works despite quoting and will execute the echos
Obsessing over POSIX purity is a pointless endeavor. It should stay in its corner, but is largely a waste of time outside of corner cases. Do not mix the two and understand the differences, but remaining locked inside a cage you make for yourself is a pointless show. The real world has bash everywhere that matters.
PS: Be good, or I'll replace you with a very small zsh script. ;) Interestingly, it possible to dynamically add additional native commands to both zsh and bash in other languages. LuaBash and bash-loadables are examples.
In many cases, the fact that it's far from trivial to figure out whether your script is portable or not, makes them not portable for practical intents and purposes. I would say that for most people and most use cases, shell scripts are write once and hope you never have to edit them again, and even knowing one way of comparing values may be problematic while another might not be is beyond the given pay grade.
You can of course test portability by testing, or simply writing, your script against a POSIX-only shell interpretation. It's a lot easier to change a shebang (#!/bin/ash -> #!/bin/sh) than to rewrite the full script during a firefight.
One might even suggest "/bin/bash considered harmful".
One also may suggest "/bin/sh considered harmful". Does the rest of the world have to suffer just because Solaris can't be arsed to upgrade its tooling (much of it is very tentatively POSIX-compliant anyhow) so they block any new additions to the /bin/sh features in the POSIX?
The dash shell compiles to 80k on an i386, which is viable for embedded applications.
The POSIX shell was standardized in view of the original Korn shell, which could compile to 64k on Xenix 286.
The functionality was reduced in an effort to increase code maintainability.
Edit:
"A lot of effort was made to keep ksh88 small. In fact the size you report on Solaris is without stripping the symbol table. The size that I am getting for ksh88i on Solaris is 160K and the size on NetBSD on intel is 135K.
"ksh88 was able to compile on machines that only allowed 64K text. There were many compromises to this approach. I gave up on size minimization with ksh93."
This can be said of most programming, in Java, C++, golang, Pascal and Ruby I have been on projects that could not compile. The dependencies are so out dated a complete rewrite is easier. I do not find bash worse or better. You just do alot more with less lines in it.
I believe many Linuxes still symlink /bin/sh -> /bin/bash, so the shebang itself won't save you. Debian's assignment of /bin/dash as the default /bin/sh is relatively recent (at least for those who've been running Debian since the 1990s).
I virtually exclusively write Bash scripts for my own use. I've been bit by bashisms however, and at the very least don't casually dismiss the concerns which are voiced against doing this. I'm also daily using systems which don't have a full Bash, and which really don't have the resources to run it (mostly storage, though other factors may be involved).
I've got sympathies with both sides of this argument.
Bashisms are powerful, flexible, and occasionally performant.
They're also non-portable, and you're apt to discover this whilst in an emergency situation with limited resources, most of all time. Having to edit shell scripts into POSIX conformance on a smartphone SSH session is No Bueno.
There are an awful lot of systems which have either no, or old, Bash. Most prevalent of the latter is MacOS / OSX, which is now nagging me on each shell initialisation that the current supported standard is not bash but zsh. Scripts linked to /bin/bash will of course run, but the last update was version 3.2.57(1) if my system reports correctly. The most recent GNU Bash release is 5.3 according to <https://ftp.gnu.org/gnu/bash/>. And that's hardly the only old Bash floating around.
On many systems, including many routers, DSL/cable modems, and other embedded devices, shell is some alternate variant (dash is /bin/sh on Debian, ash on RHEL's initrd IIRC, which last I fought with it was a script-only interpreter, not capable of running interactive commands, fucking annoying as hell). And there are still legacy Unix systems running Bourne, ksh, or other shells. Not to mention the phenomenal number of systems for which busybox provides much if not all of the userland environment, including /bin/sh. And is not bashism-capable.
Ignoring POSIX works fantastically until it doesn't. Caveat scriptor.
It's fine to write a bash script, if you know you'll have bash. It's not great to write a bash script with #!/bin/sh It's also not great to write a bash script when you have to run without bash.
>You will understand why when you try this with /bin/sh on Debian/Ubuntu.
Okay so just ... don't do that? You're meant to put /bin/bash in the shebang ... because it's a bash script. I don't understand this self-inflicted problem; obviously if you run a bash script with sh, it won't do what you want! They're different things! You may as well complain it won't run under /bin/python.
If you pull the HTML file off the above link, you will get a directory listing of all of the POSIX.2 utilities (tar, sed, awk, etc.). The options that you find in the specifications are maximally portable.
>However, the value was mostly gone by the mid-to-late 1990s, and the few remaining issues were cleaned up before 2010 — shockingly late, but still over a decade ago.
This is an argument for it: You never know when you'll be on a legacy system. "A decade" isn't that long.
I feel like something's missing here. I recall having to resort to the x-hack not that many years ago to acommodate for compatibility with a system shell somewhere. Beats me if I recall which and where but I'm pretty sure it had to do with empty strings.
It does nothing to help with quoting, if $var is broken then so is x$var. About half the usages I've found (and this is some pretty old code) quote the variables correctly.
The assumption for this use was var was a valid single word but could conflict with switches to [ / test or it could be the empty string. If var was already guaranteed to be [^[:space:][:punct:][:cntrl:]]+, then x would also have been unnecessary too.
[ wasn't an operator or language construct but an external program similar or the same as test. ] was a final argument.
In today's world where bash is and does supplant POSIX for all intents and purposes, use [[ and (( because they're part of the interpreter and more flexible.
When I need to write shell scripts, it's because I need portability, so I write POSIX shell scripts. If I can compromise on portability anyway, I jump right past Bash to a more sane general purpose language, like Python.
Until you realize that some of the expressions in [[ are evaluated as arithmetic expression which has some surprises around array subscript and command substitution.
In short, because the following works despite quoting and will execute the echos
I'm back to use the single [ for the arithmetic binary operators.No, don't do that.
You will understand why when you try this with /bin/sh on Debian/Ubuntu.
This incompatibility was introduced for reasons of speed and standards compliance.
There is a time and a place to rely on advanced shell features, but it should not be the default preference.
> You will understand why when you try this with /bin/sh on Debian/Ubuntu.
So instead of having your shebang be /bin/sh, make it /usr/bin/env bash.
Obsessing over POSIX purity is a pointless endeavor. It should stay in its corner, but is largely a waste of time outside of corner cases. Do not mix the two and understand the differences, but remaining locked inside a cage you make for yourself is a pointless show. The real world has bash everywhere that matters.
PS: Be good, or I'll replace you with a very small zsh script. ;) Interestingly, it possible to dynamically add additional native commands to both zsh and bash in other languages. LuaBash and bash-loadables are examples.
Advanced shell features will not run with dash, which is (almost) strict POSIX.
They also fail with busybox.
There are many subtle differences between BASH and Korn, so much will not run without rewrite on Android's mksh.
Because you have not needed portable scripts yourself does not imply that the need is not vast.
In many cases, the fact that it's far from trivial to figure out whether your script is portable or not, makes them not portable for practical intents and purposes. I would say that for most people and most use cases, shell scripts are write once and hope you never have to edit them again, and even knowing one way of comparing values may be problematic while another might not be is beyond the given pay grade.
You can of course test portability by testing, or simply writing, your script against a POSIX-only shell interpretation. It's a lot easier to change a shebang (#!/bin/ash -> #!/bin/sh) than to rewrite the full script during a firefight.
One might even suggest "/bin/bash considered harmful".
One also may suggest "/bin/sh considered harmful". Does the rest of the world have to suffer just because Solaris can't be arsed to upgrade its tooling (much of it is very tentatively POSIX-compliant anyhow) so they block any new additions to the /bin/sh features in the POSIX?
> One also may suggest "/bin/sh considered harmful".
Then don't use /sbin/sh?
If you want bashisms, just use /usr/bin/env bash.
<https://news.ycombinator.com/item?id=41682985>
It's hardly just Solaris.
The dash shell compiles to 80k on an i386, which is viable for embedded applications.
The POSIX shell was standardized in view of the original Korn shell, which could compile to 64k on Xenix 286.
The functionality was reduced in an effort to increase code maintainability.
Edit:
"A lot of effort was made to keep ksh88 small. In fact the size you report on Solaris is without stripping the symbol table. The size that I am getting for ksh88i on Solaris is 160K and the size on NetBSD on intel is 135K.
"ksh88 was able to compile on machines that only allowed 64K text. There were many compromises to this approach. I gave up on size minimization with ksh93."
https://m.slashdot.org/story/16351
This can be said of most programming, in Java, C++, golang, Pascal and Ruby I have been on projects that could not compile. The dependencies are so out dated a complete rewrite is easier. I do not find bash worse or better. You just do alot more with less lines in it.
If you use arrays, you cannot use dash.
The correct solution here is to pick one of the following options:
1) Write a bash script and put /bin/bash in your shebang
2) Write a POSIX shell script and integrate checkbashisms into your editor.
Neither is more correct than the other.
I believe many Linuxes still symlink /bin/sh -> /bin/bash, so the shebang itself won't save you. Debian's assignment of /bin/dash as the default /bin/sh is relatively recent (at least for those who've been running Debian since the 1990s).
I virtually exclusively write Bash scripts for my own use. I've been bit by bashisms however, and at the very least don't casually dismiss the concerns which are voiced against doing this. I'm also daily using systems which don't have a full Bash, and which really don't have the resources to run it (mostly storage, though other factors may be involved).
3) #!/bin/dash
#!/usr/bin/env dash
I mean, if we're targeting portability...
Classic IAGNI/YAGNI conflation.
I've got sympathies with both sides of this argument.
Bashisms are powerful, flexible, and occasionally performant.
They're also non-portable, and you're apt to discover this whilst in an emergency situation with limited resources, most of all time. Having to edit shell scripts into POSIX conformance on a smartphone SSH session is No Bueno.
There are an awful lot of systems which have either no, or old, Bash. Most prevalent of the latter is MacOS / OSX, which is now nagging me on each shell initialisation that the current supported standard is not bash but zsh. Scripts linked to /bin/bash will of course run, but the last update was version 3.2.57(1) if my system reports correctly. The most recent GNU Bash release is 5.3 according to <https://ftp.gnu.org/gnu/bash/>. And that's hardly the only old Bash floating around.
On many systems, including many routers, DSL/cable modems, and other embedded devices, shell is some alternate variant (dash is /bin/sh on Debian, ash on RHEL's initrd IIRC, which last I fought with it was a script-only interpreter, not capable of running interactive commands, fucking annoying as hell). And there are still legacy Unix systems running Bourne, ksh, or other shells. Not to mention the phenomenal number of systems for which busybox provides much if not all of the userland environment, including /bin/sh. And is not bashism-capable.
Ignoring POSIX works fantastically until it doesn't. Caveat scriptor.
If you want the most powerful shell, ksh93 has the most features without doubt.
Since David Korn retired, it has been adrift with few updates.
The dash shell can be compiled with libedit to implement "set -o vi" (which is also POSIX), and is pleasant to use this way.
It's fine to write a bash script, if you know you'll have bash. It's not great to write a bash script with #!/bin/sh It's also not great to write a bash script when you have to run without bash.
>You will understand why when you try this with /bin/sh on Debian/Ubuntu.
Okay so just ... don't do that? You're meant to put /bin/bash in the shebang ... because it's a bash script. I don't understand this self-inflicted problem; obviously if you run a bash script with sh, it won't do what you want! They're different things! You may as well complain it won't run under /bin/python.
If you use #!/bin/sh then you must understand the POSIX.2 shell standard.
If you use arrays, coprocesses, or advanced conditional [[ then your script will fail.
The rules for #!/bin/sh as POSIX.2 are here:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
Will this fail on rhel? No, as bash implements #!/bin/sh. Will it fail on Ubuntu? Absolutely.
When I benchmarked it, I saw no difference between [ and [[ when using bash, so I assume [ is handled (now) by the shell as well.
The syntax inside [[ is different though.
The dash shell is supposedly four times faster than bash, and dramatically reduced Ubuntu's boot time.
The [[ test syntax is not implemented in dash. It is listed as a future reserved word in the POSIX shell standard, but is not (yet?) specified.
(Google: POSIX shell)
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...
If you pull the HTML file off the above link, you will get a directory listing of all of the POSIX.2 utilities (tar, sed, awk, etc.). The options that you find in the specifications are maximally portable.
>However, the value was mostly gone by the mid-to-late 1990s, and the few remaining issues were cleaned up before 2010 — shockingly late, but still over a decade ago.
This is an argument for it: You never know when you'll be on a legacy system. "A decade" isn't that long.
I feel like something's missing here. I recall having to resort to the x-hack not that many years ago to acommodate for compatibility with a system shell somewhere. Beats me if I recall which and where but I'm pretty sure it had to do with empty strings.
Of course, I am now wondering why people are using /bin/[ in a shell script in 2024?
Because it makes a lot of sense in environments where sysadmins like to know what is going on.
Using /bin keeps a system in sync with the platform distribution. Keeping things tied to /bin should work between upgrade cycles.
But are there even "sysadmins" anymore?
Somewhere I would like to believe there are. I certainly identify as sysadmin.
Nobody used it with the quotes. The point was to get rid of the quotes.
[ x$var = xval ]
It does nothing to help with quoting, if $var is broken then so is x$var. About half the usages I've found (and this is some pretty old code) quote the variables correctly.
The assumption for this use was var was a valid single word but could conflict with switches to [ / test or it could be the empty string. If var was already guaranteed to be [^[:space:][:punct:][:cntrl:]]+, then x would also have been unnecessary too.
If we have set var='hello world' then the syntax breaks.
This is discussed in the article.