updated descriptions, added more exercises

Author: learnbyexample

command-line-overview.html

@@ -48,7 +48,7 @@
  6 25 greeting.txt
 </code></pre><p>Filename won't be printed for stdin data. This is helpful to save the results in a variable for scripting purposes.<pre><code class=language-bash>$ wc -l &LTgreeting.txt
 2
-</code></pre><p>Word count is based on whitespace separation. You'll have to pre-process the input if you do not want certain non-whitespace characters to influence the results. <code>tr</code> can be used to remove a particular set of characters (this command will be discussed in <a href=/assorted-text-processing-tools.html>Assorted Text Processing Tools</a> chapter).<pre><code class=language-bash>$ echo 'apple ; banana ; cherry' | wc -w
+</code></pre><p>Word count is based on whitespace separation. You'll have to pre-process the input if you do not want certain non-whitespace characters to influence the results. <code>tr</code> can be used to remove a particular set of characters (this command will be discussed in the <a href=/assorted-text-processing-tools.html>Assorted Text Processing Tools</a> chapter).<pre><code class=language-bash>$ echo 'apple ; banana ; cherry' | wc -w
 5
 
 # remove characters other than alphabets and whitespace
@@ -180,7 +180,7 @@
 $ touch -m -d '2000-01-01 00:00:01' hi.sh
 $ stat -c '%y' hi.sh
 2000-01-01 00:00:01.000000000 +0530
-</code></pre><p>As seen in <a href=./managing-files-directories.html>Managing Files and Directories</a> chapter, <code>touch</code> creates a new file if the target file doesn't exist yet. You can use the <code>-c</code> option to prevent this behavior.<pre><code class=language-bash>$ ls report.txt
+</code></pre><p>As seen in the <a href=./managing-files-directories.html>Managing Files and Directories</a> chapter, <code>touch</code> creates a new file if the target file doesn't exist yet. You can use the <code>-c</code> option to prevent this behavior.<pre><code class=language-bash>$ ls report.txt
 ls: cannot access 'report.txt': No such file or directory
 $ touch report.txt
 $ ls report.txt
@@ -344,38 +344,38 @@
 7.7M    report.log
 8.2k    todos
 7.8M    total
-</code></pre><p><strong>8)</strong> When will you use the <code>df</code> command instead of <code>du</code>? Which <code>df</code> command option will help you to report only specific fields of interest?<p><strong>9)</strong> Display the size of <code>scores.csv</code> and <code>timings.txt</code> files in the format shown below.<pre><code class=language-bash>$ stat # ???
+</code></pre><p><strong>8)</strong> What does the <code>du --apparent-size</code> option do?<p><strong>9)</strong> When will you use the <code>df</code> command instead of <code>du</code>? Which <code>df</code> command option will help you to report only specific fields of interest?<p><strong>10)</strong> Display the size of <code>scores.csv</code> and <code>timings.txt</code> files in the format shown below.<pre><code class=language-bash>$ stat # ???
 scores.csv: 70
 timings.txt: 49
-</code></pre><p><strong>10)</strong> Which <code>touch</code> option will help you prevent file creation if it doesn't exist yet?<p><strong>11)</strong> Assume <code>new_file.txt</code> doesn't exist in the current working directory. What would be the output of the <code>stat</code> command shown below?<pre><code class=language-bash>$ touch -t '202010052010.05' new_file.txt
+</code></pre><p><strong>11)</strong> Which <code>touch</code> option will help you prevent file creation if it doesn't exist yet?<p><strong>12)</strong> Assume <code>new_file.txt</code> doesn't exist in the current working directory. What would be the output of the <code>stat</code> command shown below?<pre><code class=language-bash>$ touch -t '202010052010.05' new_file.txt
 $ stat -c '%y' new_file.txt
 # ???
-</code></pre><p><strong>12)</strong> Is the following <code>touch</code> command valid? If so, what would be the output of the <code>stat</code> command that follows?<pre><code class=language-bash># change to the 'scripts' directory and source the 'touch.sh' script
+</code></pre><p><strong>13)</strong> Is the following <code>touch</code> command valid? If so, what would be the output of the <code>stat</code> command that follows?<pre><code class=language-bash># change to the 'scripts' directory and source the 'touch.sh' script
 $ source touch.sh
 
 $ touch -r fruits.txt f{1..3}.txt
 $ stat -c '%n: %y' f*.txt
 # ???
-</code></pre><p><strong>13)</strong> Use appropriate option(s) to get the output shown below.<pre><code class=language-bash>$ printf 'αλεπού\n' | file -
+</code></pre><p><strong>14)</strong> Use appropriate option(s) to get the output shown below.<pre><code class=language-bash>$ printf 'αλεπού\n' | file -
 /dev/stdin: UTF-8 Unicode text
 
 $ printf 'αλεπού\n' | file # ???
 UTF-8 Unicode text
-</code></pre><p><strong>14)</strong> Is the following command valid? If so, what would be the output?<pre><code class=language-bash>$ basename -s.txt ~///test.txt///
+</code></pre><p><strong>15)</strong> Is the following command valid? If so, what would be the output?<pre><code class=language-bash>$ basename -s.txt ~///test.txt///
 # ???
-</code></pre><p><strong>15)</strong> Given the file path in the shell variable <code>p</code>, how'd you obtain the output shown below?<pre><code class=language-bash>$ p='~/projects/square_tictactoe/python/game.py'
+</code></pre><p><strong>16)</strong> Given the file path in the shell variable <code>p</code>, how'd you obtain the output shown below?<pre><code class=language-bash>$ p='~/projects/square_tictactoe/python/game.py'
 $ dirname # ???
 ~/projects/square_tictactoe
-</code></pre><p><strong>16)</strong> Explain what each of the characters mean in the following output.<pre><code class=language-bash>$ stat -c '%A' ../scripts/
+</code></pre><p><strong>17)</strong> Explain what each of the characters mean in the following output.<pre><code class=language-bash>$ stat -c '%A' ../scripts/
 drwxrwxr-x
-</code></pre><p><strong>17)</strong> What would be the output of the second <code>stat</code> command shown below?<pre><code class=language-bash>$ touch new_file.txt
+</code></pre><p><strong>18)</strong> What would be the output of the second <code>stat</code> command shown below?<pre><code class=language-bash>$ touch new_file.txt
 $ stat -c '%a %A' new_file.txt
 664 -rw-rw-r--
 
 $ chmod 546 new_file.txt
 $ stat -c '%a %A' new_file.txt
 # ???
-</code></pre><p><strong>18)</strong> How would you specify directory permissions using the <code>mkdir</code> command?<pre><code class=language-bash># instead of this
+</code></pre><p><strong>19)</strong> How would you specify directory permissions using the <code>mkdir</code> command?<pre><code class=language-bash># instead of this
 $ mkdir back_up
 $ chmod 750 back_up
 $ stat -c '%a %A' back_up
@@ -386,14 +386,14 @@
 $ mkdir # ???
 $ stat -c '%a %A' back_up
 750 drwxr-x---
-</code></pre><p><strong>19)</strong> Change the file permission of <code>book_list.txt</code> to match the output of the second <code>stat</code> command shown below. Don't use the number <code>220</code>, specify the changes in terms of <code>rwx</code> characters.<pre><code class=language-bash>$ touch book_list.txt
+</code></pre><p><strong>20)</strong> Change the file permission of <code>book_list.txt</code> to match the output of the second <code>stat</code> command shown below. Don't use the number <code>220</code>, specify the changes in terms of <code>rwx</code> characters.<pre><code class=language-bash>$ touch book_list.txt
 $ stat -c '%a %A' book_list.txt
 664 -rw-rw-r--
 
 # ???
 $ stat -c '%a %A' book_list.txt
 220 --w--w----
-</code></pre><p><strong>20)</strong> Change the permissions of <code>test_dir</code> to match the output of the second <code>stat</code> command shown below. Don't use the number <code>757</code>, specify the changes in terms of <code>rwx</code> characters.<pre><code class=language-bash>$ mkdir test_dir
+</code></pre><p><strong>21)</strong> Change the permissions of <code>test_dir</code> to match the output of the second <code>stat</code> command shown below. Don't use the number <code>757</code>, specify the changes in terms of <code>rwx</code> characters.<pre><code class=language-bash>$ mkdir test_dir
 $ stat -c '%a %A' test_dir
 775 drwxrwxr-x
 

file-properties.html

@@ -252,7 +252,7 @@
 0
 </code></pre><p>As seen in the examples above, you can check the exit status of the last executed command using the <code>$?</code> special variable. <code>0</code> means everything went well and higher numbers indicate some sort of failure has occurred (the details of which you can look up in the command's manual).<blockquote><p><img alt=info src=images/info.svg> <img alt=warning src=images/warning.svg> Linux filenames can use any character other than <code>/</code> and the ASCII NUL character. Quote the arguments if it contains characters like space, <code>*</code>, etc to prevent shell expansion. Shell considers space as the argument separator, <code>*</code> is a wildcard character and so on. As a good practice, use only alphabets, numbers and underscores for filenames, unless you have some specific requirements. See also <a href=https://unix.stackexchange.com/q/269093/109046>unix.stackexchange: Characters best avoided in filenames</a>.</blockquote><blockquote><p><img alt=info src=./images/info.svg> You can delete the practice directory if you wish:<pre><code class=language-bash>$ cd ..
 $ rm -r practice_mkdir
-</code></pre></blockquote><h2 id=touch><a class=header href=#touch>touch</a></h2><p>You'll usually create files using a text editor or by redirecting output of a command to a file. For some cases, empty files are needed for testing purposes or to satisfy a particular build process. A real world use case is the empty <code>.nojekyll</code> file for <a href=https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages#static-site-generators>GitHub Pages</a>.<p>The <code>touch</code> command's main functionality is altering timestamps (which will be discussed in <a href=./file-properties.html>File Properties</a> chapter). If a file doesn't exist, <code>touch</code> will create an empty file using the current timestamp. You can also pass more than one file argument if needed.<pre><code class=language-bash>$ mkdir practice_touch
+</code></pre></blockquote><h2 id=touch><a class=header href=#touch>touch</a></h2><p>You'll usually create files using a text editor or by redirecting output of a command to a file. For some cases, empty files are needed for testing purposes or to satisfy a particular build process. A real world use case is the empty <code>.nojekyll</code> file for <a href=https://docs.github.com/en/pages/getting-started-with-github-pages/about-github-pages#static-site-generators>GitHub Pages</a>.<p>The <code>touch</code> command's main functionality is altering timestamps (which will be discussed in the <a href=./file-properties.html>File Properties</a> chapter). If a file doesn't exist, <code>touch</code> will create an empty file using the current timestamp. You can also pass more than one file argument if needed.<pre><code class=language-bash>$ mkdir practice_touch
 $ cd practice_touch
 
 $ ls ip.txt

managing-files-directories.html

@@ -28,7 +28,7 @@
                     document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
                     Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
                         link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
-                    });</script><div class=content id=content><main><div class=sidetoc><nav class=pagetoc></nav></div><h1 id=multipurpose-text-processing-tools><a class=header href=#multipurpose-text-processing-tools>Multipurpose Text Processing Tools</a></h1><p>Many CLI text processing tools have been in existence for about half a century. And newer tools are being written to solve ever expanding text processing problems. Just knowing that a particular tool exists or searching for a tool before attempting to write your own solution can be a time saver. Also, popular tools are likely to be optimized for speed, hardened against bugs from wide usage, discussed on forums, and so on.<p><code>grep</code> was already covered in <a href=./searching-files-and-filenames.html>Searching Files and Filenames</a> chapter. In addition, <code>sed</code>, <code>awk</code> and <code>perl</code> are essential tools to solve a wide variety of text processing problems from the command line. In this chapter, you'll learn field processing, use regular expressions for search and replace requirements, perform operations based on multiple lines and files, etc.<blockquote><p><img alt=info src=./images/info.svg> Examples presented in this chapter only cover some of the functionalities. I've written separate books to cover such tools with more detailed explanations, examples and exercises. See <a href=https://learnbyexample.github.io/books/>https://learnbyexample.github.io/books/</a> for links to these books.</blockquote><blockquote><p><img alt=info src=./images/info.svg> The <a href=https://github.com/learnbyexample/cli-computing/tree/master/example_files>example_files</a> directory has the sample input files used in this chapter. Change to the <code>example_files/text_files</code> directory to follow along the examples shown in this chapter.</blockquote><h2 id=sed><a class=header href=#sed>sed</a></h2><p>The command name <code>sed</code> is derived from <strong>s</strong>tream <strong>ed</strong>itor. Here, stream refers to data being passed via shell pipes. Thus, the command's primary functionality is to act as a text editor for <strong>stdin</strong> data with <strong>stdout</strong> as the output target. You can also edit file input and save the changes back to the same file if needed.<h3 id=substitution><a class=header href=#substitution>Substitution</a></h3><p><code>sed</code> has various commands to manipulate text input. <strong>substitute</strong> command is most commonly used, whose syntax is <code>s/REGEXP/REPLACEMENT/FLAGS</code>. Here are some examples:<pre><code class=language-bash># for each input line, change only the first ',' to '-'
+                    });</script><div class=content id=content><main><div class=sidetoc><nav class=pagetoc></nav></div><h1 id=multipurpose-text-processing-tools><a class=header href=#multipurpose-text-processing-tools>Multipurpose Text Processing Tools</a></h1><p>Many CLI text processing tools have been in existence for about half a century. And newer tools are being written to solve ever expanding text processing problems. Just knowing that a particular tool exists or searching for a tool before attempting to write your own solution can be a time saver. Also, popular tools are likely to be optimized for speed, hardened against bugs from wide usage, discussed on forums, and so on.<p><code>grep</code> was already covered in the <a href=./searching-files-and-filenames.html>Searching Files and Filenames</a> chapter. In addition, <code>sed</code>, <code>awk</code> and <code>perl</code> are essential tools to solve a wide variety of text processing problems from the command line. In this chapter, you'll learn field processing, use regular expressions for search and replace requirements, perform operations based on multiple lines and files, etc.<blockquote><p><img alt=info src=./images/info.svg> Examples presented in this chapter only cover some of the functionalities. I've written separate books to cover such tools with more detailed explanations, examples and exercises. See <a href=https://learnbyexample.github.io/books/>https://learnbyexample.github.io/books/</a> for links to these books.</blockquote><blockquote><p><img alt=info src=./images/info.svg> The <a href=https://github.com/learnbyexample/cli-computing/tree/master/example_files>example_files</a> directory has the sample input files used in this chapter. Change to the <code>example_files/text_files</code> directory to follow along the examples shown in this chapter.</blockquote><h2 id=sed><a class=header href=#sed>sed</a></h2><p>The command name <code>sed</code> is derived from <strong>s</strong>tream <strong>ed</strong>itor. Here, stream refers to data being passed via shell pipes. Thus, the command's primary functionality is to act as a text editor for <strong>stdin</strong> data with <strong>stdout</strong> as the output target. You can also edit file input and save the changes back to the same file if needed.<h3 id=substitution><a class=header href=#substitution>Substitution</a></h3><p><code>sed</code> has various commands to manipulate text input. <strong>substitute</strong> command is most commonly used, whose syntax is <code>s/REGEXP/REPLACEMENT/FLAGS</code>. Here are some examples:<pre><code class=language-bash># for each input line, change only the first ',' to '-'
 $ printf '1,2,3,4\na,b,c,d\n' | sed 's/,/-/'
 1-2,3,4
 a-b,c,d