-
Notifications
You must be signed in to change notification settings - Fork 5
/
group1.html
307 lines (225 loc) · 15.1 KB
/
group1.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
<!doctype html>
<html lang="en">
<head>
<title>Cron: Automate and Schedule Scripts</title>
<meta charset="utf-8" />
<meta name="keywords" content="" />
<meta name="description" content="" />
<link rel="stylesheet" href="style1.css" type="text/css" media="screen" />
</head>
<body>
<div id="logo">
<h1>ITC134 FL16 LINUX Shell Resource</h1>
</div>
<hr />
<header>
<div id="menu">
<ul>
<li class="inactive"><a href="index.html">Home</a></li>
<li class="current_page_item"><a href="group1.html">Group 1</a></li>
<li class="inactive"><a href="group2.html">Group 2</a></li>
<li class="inactive"><a href="group3.html">Group 3</a></li>
<li class="inactive"><a href="group4.html">Group 4</a></li>
<li class="inactive"><a href="group5.html">Group 5</a></li>
</ul>
</div>
</header>
<div id="page">
<div id="content">
<div class="post">
<h2 class="title"><a href="#">Cron: Automate and Schedule Scripts</a></h2>
<div class="entry">
<h1>What is cron?</h1>
<p>Cron jobs are simply bash scripts that run when you boot up your computer and when certain time conditions have been met. For them, you write your script, save it, and then run a command to add a new cron job that points to the location of the newly-saved script.</p>
<h2>What are they good for?</h2>
<p>Cron is a Linux/Unix utility, which schedules a command or script on your server to run automatically at a specified time and date. A cron job is the scheduled task itself. Cron jobs can be very useful to automate repetitive tasks.</p>
<p>For example, you can set a cron job to delete temporary files every week to conserve your disk space. Some programs, such as Drupal, may even require you to set up a cron job to perform certain functions.</p>
<p>Also, you might use a cron job to send out an email report on a daily basis, or to update some cached data every 10 minutes, or refresh summary information once an hour.</p>
<p>Scripts executed as a cron job are typically used to modify files or databases. However, they can perform other tasks that do not modify data on the server, like sending out email notifications.</p>
<h2>Who typically uses them?</h2>
<ul>
<li>System Administrators</li>
<li>Database Administrators</li>
<li>Power Users</li>
<li>Developers</li>
</ul>
<h2>Elements of a Cron Job</h2>
<h3>Most cron jobs include three components:</h3>
<ul>
<li>The script that is to be called or executed.</li>
<li>The command that executes the script on a recurring basis (typically set in the Panel).</li>
<li>The action or output of the script (which depends on what the script being called does).</li>
</ul>
<h3>Cron Job Limits</h3>
<ul>
<li>You may NOT run a cron job more often than every minute.</li>
<li>It isn't suitable for complex, event-driven tasks.</li>
<li>Not flexible for running tasks.</li>
<li>No queueing for cron: no specific ordering of jobs.</li>
<li>Won't be able to protect resources from having too many things running out of it at once.</li>
</ul>
<p><strong><i>Let's get started!</i></strong></p>
<p>In this example we have a script that resizes images and a cron job that schedules and executes the script.</p>
<p>First, if you don't already have it, you will need to install the command line utility suite ImageMagick:</p>
<pre>
# sudo is the command that tells Linux to run the following command as root
# apt is the Advanced Packaging Tool, and apt-get tells it to find and download the package that is named
# install tells it to install the package after downloading
# imagemagick is the name of the package
sudo apt-get install imagemagick
</pre>
<pre>
# here follows simple commands to convert and resize files with ImageMagick
# converts the png "spoon" in the images-raw directory into a jpg file with a 95% compression
convert ~/images_raw/spoon.png -quality 95 spoon.jpg
# resizes the jpg "spork" in the images-raw directory to a height of 500 pixels, maintaining aspect ratio
convert ~/images_raw/spork.jpg -resize x500 spork.jpg
</pre>
<p>But what we really want here is a script that we can schedule to run at regular intervals, and we need it to do the following things:</p>
<ol>
<li>Grab a png image</li>
<li>Save the name from the png to give to the jpg</li>
<li>Convert the png to a jpg</li>
<li>Move the new file to a different directory</li>
<li>Remove the old png file</li>
</ol>
<p>We're also going to have it output the old file name and new file name so we can see it's working. Here's what a simple version of that looks like:</p>
<pre>
#!/bin/bash is the standard way of commenting to say this is a bash script
#cd is change directory, and then the directory you want it to work in
cd ~/workspace/images_raw/
# begins a for loop
# for every file name that ends in .png
for FILENAME in *.png
# do the following
do
# create a variable called NEWFILENAME and make it the old FILENAME
# with .jpg substituted for .png
# See documentation below for an explanation of Shell Parameter Expansion
NEWFILENAME="${FILENAME/%.png/.jpg}"
# this prints to the screen (echo) while allowing escape characters like tab (-e)
# the old file name and the new file name, so you can see that it's working
# \t is a tab, so that the old file name and new file name line up neatly
echo -e "Old name: $FILENAME\t" "New name: $NEWFILENAME"
# this is the convert command we used above, converting the file with the old name
# to the jpg file and giving it the new name
# && says that if and only if the previous command ran properly, do the next thing
# \ lets you line return in the middle of a line of code to make it more readable
convert "$FILENAME" "$NEWFILENAME" && \
# move the thing called NEWFILENAME to the converted_images folder
mv "$NEWFILENAME" ~/workspace/converted_images/ && \
# remove the old file
rm "$FILENAME"
# done ends the do command, defining the point at which a single loop stops
# and then, because this is a loop, it does it to every file that ends in .png in the directory
done
</pre>
<p>And here's what the code looks like without all the comments. It's really very short.</p>
<pre>
cd ~/workspace/images_raw/
for FILENAME in *.png
do
NEWFILENAME="${FILENAME/%.png/.jpg}"
echo -e "Old name: $FILENAME\t" "New name: $NEWFILENAME"
convert "$FILENAME" "$NEWFILENAME" && mv "$NEWFILENAME" ~/workspace/converted_images/ && rm "$FILENAME"
done
</pre>
<p>Paste that into a text file called convert_script.sh, save it in your home directory, and type bash convert_script.sh into the command line. You should see something like this:</p>
<pre>
<beforerun.jpg>
<afterrun.jpg>
</pre>
<p>You can see in the first picture that the files on the left sidebar are in the images_raw directory, and then in the second picture the new jpgs are in the converted_images directory. You can also see in the console that it lists off the file names as it runs through the loop.</p>
<p>There are always refinements you can add to it, extra things that will double check to make sure the new files really are jpgs, or tests for errors. But this is a basic script.</p>
<h2>Documentation for 3.5.3 Shell Parameter Expansion:</h2>
<blockquote>
<p>
${parameter/pattern/string}<br />
The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. If pattern begins with '/', all matches of pattern are replaced with string. Normally only the first match is replaced. If pattern begins with '#', it must match at the beginning of the expanded value of parameter. If pattern begins with '%', it must match at the end of the expanded value of parameter. If string is null, matches of pattern are deleted and the / following pattern may be omitted. If the nocasematch shell option (see the description of shopt in The Shopt Builtin) is enabled, the match is performed without regard to the case of alphabetic characters. If parameter is '@' or '*', the substitution operation is applied to each positional parameter in turn, and the expansion is the resultant list. If parameter is an array variable subscripted with '@' or '*', the substitution operation is applied to each member of the array in turn, and the expansion is the resultant list <br />
-From <a href="https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html" target="_blank">the bash manual</a>.
</p>
</blockquote><br />
<p>So <b>NEWFILENAME="${FILENAME/%.png/.jpg}"</b> says that the pattern for NEWFILENAME is to take a FILENAME that ends in .png and to copy it but replace .png with .jpg.</p>
<p>Now we just need to schedule the cron job!</p>
<p>First, open a new file in a text editor -- make sure it's a text editor and not a word processor -- and start with a MAILTO command. MAILTO tells crontab you want to be emailed with output from the job. On the next line, we'll tell it to discard everything but error messages, so that you only get notified if something goes wrong. A MAILTO command is just [email protected], with whatever your actual email is.</p>
<p>On a new line (very important), you're going to specify WHEN to schedule it. The first place is minutes (00-60), the second is hours (00-24, on 24-hour time, not 12-hour), the third is days (00-31), the fourth is months (00-12), and the fifth is day of the week (0-7). You can tell it to run every hour at fifteen minutes past the hour, every day at 12:15am, the first of every month, the fifth of May every year, or every Tuesday and Thursday, just for a few examples. Commas separate different scheduling within the same time from (00,30 in the minutes slot for on the hour and half hour, eg, or 3,5 in the days of the week slot for Tuesday and Thursday), and asterisks (*) says every instance of that slot (so, 00,30 * for every hour on the hour and half hour). Then you designate the path of the script to be run. This is followed by any other commands you want it to run when it runs your script. The commands MUST be on the same line as the scheduling. We're going to tell it to send all normal outputs to dev/null (which is like a trash can that's immediately emptied, or a black hole). Error messages, not being normal outputs, will go to the email address provided.</p>
<p>End your file with a new line, always.</p>
<p>We're going to schedule ours at 11:59pm every night. It looks like this:</p>
<pre>
# the MAILTO line tells it to email you any output that is not otherwise directed
# this says that at 59 minutes past the 23rd hour (11pm)
# on every day of every month of every year
# so every day at 11:59pm
# to run the script at ~/workspace/convert.sh
# all output EXCEPT error messages will go to dev/null, which is like the trash
# error messages go to your email
59 23 * * * ~/workspace/convert.sh >/dev/null
#new line to end the file
</pre>
<p>Or, without comments:</p>
<pre>
59 23 * * * ~/workspace/convert.sh >/dev/null
</pre>
<p>Save the text file as cronjob.txt or something else that you'll recognize.</p>
<p>Now you need to tell crontab that your text file is something it should pay attention to. Crontab installs your scheduling into cronjob, which will then actually run it at the specified time.</p>
<p>Go to the command line and tell it to go to the directory your scheduling file is in, and then use the crontab command to tell it to pay attention to your job.</p>
<p>(The $ below is the prompt, you don't type it. Hit return between commands.)</p>
<pre>
$ cd ~/workspace
$ crontab cronjob.txt
</pre>
<p>You can also view all of your scheduled jobs with the command crontab -l (that's a lowercase L, not a 1), and remove them all with crontab -r.</p>
<p>Put all of your cron jobs into the same file. Designate your email once, at the top of the file, and then one line for each scheduled job and any commands related to it. End the file with a new line.</p>
<p>For more on cron jobs, check out The Site Wizard (<a href="https://www.thesitewizard.com/general/set-cron-job.shtml" target="_blank">https://www.thesitewizard.com/general/set-cron-job.shtml</a>).</p>
<!-- Resources -->
<h2>Learn More:</h2>
<ul>
<li><a href="http://www.unixgeeks.org/security/newbie/unix/cron-1.html" target="_blank">http://www.unixgeeks.org/security/newbie/unix/cron-1.html</a></li>
<li><a href="https://en.wikipedia.org/wiki/Cron" target="_blank">https://en.wikipedia.org/wiki/Cron</a></li>
<li><a href="https://www.thesitewizard.com/general/set-cron-job.shtml" target="blank">https://www.thesitewizard.com/general/set-cron-job.shtml</a></li>
<li><a href="https://en.wikipedia.org/wiki/Ken_Thompson" target="_blank">https://en.wikipedia.org/wiki/Ken_Thompson</a></li>
<li><a href="http://www.alleft.com/sysadmin/common-cron-mistakes/" target="_blank">http://www.alleft.com/sysadmin/common-cron-mistakes/</a></li>
<li><a href="http://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work" target="_blank">http://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work</a></li>
<li><a href="http://www.nncron.ru/help/EN/working/cron-format.htm#STARTTIME" target="_blank">http://www.nncron.ru/help/EN/working/cron-format.htm#STARTTIME</a></li>
</ul>
<!--Citations -->
<h2>Citations</h2>
<ol>
<li><a href="http://www.makeuseof.com/tag/5-beginner-linux-setup-ideas-cron-jobs-shell-scripts/" target="_blank">http://www.makeuseof.com/tag/5-beginner-linux-setup-ideas-cron-jobs-shell-scripts/</a></li>
<li><a href="https://www.helpsystems.com/resources/articles/cron-scheduling-old-fallback" target="_blank">https://www.helpsystems.com/resources/articles/cron-scheduling-old-fallback</a></li>
</ol>
</div>
</div>
<!--<div class="post">
<h2 class="title"><a href="#"></a></h2>
<p class="meta"></p>
<div class="entry">
</div>
</div>
-->
</div>
<div id="sidebar">
<h2>Resource Links: </h2>
<ul>
<li><a href="group1.html">Group1</a></li>
<li><a href="group2.html">Group2</a></li>
<li><a href="group3.html">Group3</a></li>
<li><a href="group4.html">Group4</a></li>
<li><a href="group5.html">Group5</a></li>
</ul>
</div>
<div style="clear: both;"> </div>
</div>
<footer>
<small>
© 2016, All Rights Reserved - Authored by Aliya Asken
<br />
<a href="http://validator.w3.org/check/referer" target="_blank">Valid HTML</a>
<br />
<a href="http://jigsaw.w3.org/css-validator/check?uri=referer" target="_blank">Valid CSS</a>
</small>
</footer>
</body>
</html>