subst is simple utility to replace one string (or expression) into another
in given list of files.
If you like this tool, just say thanks.
0.4.0
-
There is
sedfor example?Yes, it is. But
seduse regexps engine called "Basic Regular Expressions", or "Extended Regular Expression". PCRE (Perl Compatible Regular Expressions) used bysubstis much more widely used engine. -
So I can use Perl!
Of course you can. But not everyone know how to use Perl. I know, but
substis IMHO simpler to use.
echo 'Hello World!' | subst -s 's/Hello/Hi/' -
or:
subst -p '(192\.168)\.1\.(10)' -r '\1.0.\2' /etc/hosts
Everything is in help :) Just execute:
subst --help
Look at result:
% subst --help
usage: subst.py [-h] [-p PATTERN] [-r REPLACE] [--eval-replace] [-t]
[-s "s/PAT/REP/gixsm"] [-c COUNT] [-l] [-i]
[--pattern-dot-all] [--pattern-verbose] [--pattern-multiline]
[-u] [--encoding-input ENCODING_INPUT]
[--encoding-file ENCODING_FILE]
[--encoding-filesystem ENCODING_FILESYSTEM] [-b] [-e EXT] [-W]
[--stdin] [--stdout] [-V] [--debug] [-v]
[files [files ...]]
Replace PATTERN with REPLACE in many files.
positional arguments:
files files to parse
optional arguments:
-h, --help show this help message and exit
-p PATTERN, --pattern PATTERN
pattern to replace for. Supersede --pattern-and-
replace. Required if --replace is specified.
-r REPLACE, --replace REPLACE
replacement. Supersede --pattern-and-replace. Required
if --pattern is specified.
--eval-replace if specified, make eval data from --replace(should be
valid Python code). Ignored with --pattern-and-replace
argument.
-t, --string if specified, treats --pattern as string, not as
regular expression. Ignored with --pattern-and-replace
argument.
-s "s/PAT/REP/gixsm", --pattern-and-replace "s/PAT/REP/gixsm", --pattern-and-replace "s/PAT/REP/gixsm"
pattern and replacement in one:
s/pattern/replace/g(pattern is always regular
expression, /g is optional and stands for --count=0,
/i == --ignore-case, /s == --pattern-dot-all, /m ==
--pattern-multiline).
-c COUNT, --count COUNT
make COUNT replacements for every file (0 makes
unlimited changes, default).
-l, --linear apply pattern for every line separately. Without this
flag whole file is read into memory.
-i, --ignore-case ignore case of characters when matching
--pattern-dot-all with this flag, dot(.) character in pattern match also
new line character (see:
https://docs.python.org/3/library/re.html#re.DOTALL).
--pattern-verbose with this flag pattern can be passed as verbose(see:
https://docs.python.org/3/library/re.html#re.VERBOSE).
--pattern-multiline with this flag pattern can be passed as multiline(see:
https://docs.python.org/3/library/re.html#re.MULTILINE
).
-u, --utf8 Use UTF-8 in --encoding-input, --encoding-file and
--encoding-filesystem
--encoding-input ENCODING_INPUT
set encoding for parameters like --pattern etc
(default for your system: utf-8)
--encoding-file ENCODING_FILE
set encoding for content of processed files (default
for your system: utf-8)
--encoding-filesystem ENCODING_FILESYSTEM
set encoding for paths and filenames (default for your
system: utf-8)
-b, --no-backup don't create backup of modified files.
-e EXT, --backup-extension EXT
extension for backup files(ignore if no backup is
created), without leading dot. Defaults to: "bak".
-W, --expand-wildcards
expand wildcards (see:
https://docs.python.org/3/library/glob.html) in paths
--stdin read data from STDIN(implies --stdout)
--stdout output data to STDOUT instead of change files in-
place(implies --no-backup)
-V, --verbose show files and how many replacements was done and
short summary
--debug show more informations
-v, --version show program's version number and exit
Miscellaneous notes:
* regular expressions engine used here is PCRE, dialect from Python
* is required to pass either --pattern and -replace, or --pattern-and-
replace argument
* if pattern passed to --pattern-and-replace has /g modifier, it
overwrites --count value
* if neither /g modifier nor --count argument is passed, assume that
--count is equal 1
* if only --count is given, this value is used
* if --eval-replace is given, --replace must be valid Python code, where
can be used m variable. m holds MatchObject instance (see:
https://docs.python.org/3/library/re.html#match-objects, for example:
--eval-replace --replace 'm.group(1).lower()'
* regular expressions with non linear search read whole file to yours
computer memory - if file size is bigger then you have memory in your
computer, it fails
* parsing expression passed to --pattern-and-replace argument is very
simple - if you use / as delimiter, then in your expression can't be
used this character anymore. If you need to use same character as
delimiter and in expression, then better use --pattern and --replace
arguments
* you can test exit code to verify there was made any changes (exit code
= 0) or not (exit code = 1)
Security notes:
* be careful with --eval-replace argument. When it's given, value passed
to --replace is eval-ed, so any unsafe code will be executed!
Author:
Marcin Sztolcman <marcin@urzenia.net> // http://urzenia.net
HomePage:
http://msztolcman.github.io/subst/
Simple replace word 'Hello' with 'Hi' in data read from STDIN:
echo 'Hello World!' | subst -s 's/Hello/Hi/' -
Replace every IP address in form: 192.168.1.X (where X is few digits - single octet)
with 192.168.0.X in /etc/hosts:
subst -p '(192\.168)\.1\.(10)' -r '\1.0.\2' /etc/hosts
A.K.A. regex or regexp. You can read more on Wikipedia. Other resources:
- Python documentation on engine
substis using: (https://docs.python.org/3/library/re.html)[https://docs.python.org/3/library/re.html] - Searchable cheatsheet for Regexps: https://www.debuggex.com/cheatsheet/regex/pcre
- Regexps tester: https://www.debuggex.com/?flavor=pcre
subst should work on any platform where Python is available,
it means Linux, Windows, MacOS X etc. There is no dependencies, plain Python power :)
- Installtion using PIP
Simplest way is to use Python's built-in package system:
pip install subst
- Using sources
To install, go to GitHub releases,
download newest release, unpack and put somewhere in PATH (ie. ~/bin
or /usr/local/bin).
If You want to install newest unstable version, then just copy file to your PATH, for example:
curl https://raw.github.com/msztolcman/subst/master/subst.py > /usr/local/bin/subst
or:
wget https://raw.github.com/msztolcman/subst/master/subst.py -O /usr/local/bin/subst
Voila!
subst is tested against Python 2.7 and Python 3.3+
Marcin Sztolcman marcin@urzenia.net
If you like or dislike this software, please do not hesitate to tell me about this me via email (marcin@urzenia.net).
If you find bug or have an idea to enhance this tool, please use GitHub's issues.
The MIT License (MIT)
Copyright (c) 2013 Marcin Sztolcman
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- dropped compatibility with Python 2.6
- paths are now normalized before processing
- improvements to handling different encodings
- exit code give us info about there was any changes
- added switch --expand-wildcards (-W)
- added -V switch as an alias for --verbose
- passing invalid flags to --pattern-and-replace is now an error
- fixes and improvements in built-in help
- --pattern-* and --ignore-case was ignored for --pattern and --replace parameters
- --pattern-and-replace was incorrectly parsed with braces as delimiters
- fixed bug with changing new-line characters from dos to unix (issue #5)
- fixed bug with bad interpretation of -t param (issue #4)
- fixed bug with using subst on Windows (issue #2)
- using singular form in verbose mode when it's required
- tests are now using py.test framework, also added many new tests
- many refactorings
- improvements to pylintrc, Makefile
- using Pipenv to handling dependencies
- config for tox
- marked as compatible with Python 3.5 and 3.6
- PEP8 improvements (coding style)
- Makefile added
- improved pylintrc
- prepared and uploaded to PYPI
- typos and editorials
- better handling of non-ascii encoding in files, patterns etc
- higher priority for --pattern-* switches then modifiers in --pattern-and-replace
- unified switches syntax (was --pattern_and_replace, but other switches used dashes)
- pep8
- typos and editorials
- second public version