Replacing R/V Western Flyer with R/V David Packard in the calendar
Here is somewhat stream of consciousness notes on how to replace R/V Western Flyer with R/V David Packard in the calendar.
1. Search for all occurrences of "flyer" in the calendar software: calendar.cgi and merge_zimbra_hourly.py
➜ ~ ssh -Y pismo
Last login: Mon Feb 27 15:54:15 2023 from mbari2333.shore.mbari.org
********************************************************
********************************************************
Welcome to the
Monterey Bay Aquarium Research Institute
All use must be in accordance with
MBARI's acceptable use policy
Unauthorized Access Prohibited
********************************************************
********************************************************
[mccann@pismo ~]$ cd /var/www/html/internal/events
[mccann@pismo events]$ grep -i flyer calendar.cgi
$flyersROV = ($current_year > 2008) ? 'Doc Ricketts' : 'Tiburon';
<BR><SPAN CLASS=flyerColor>R/V <I>Western Flyer & $flyersROV</I></SPAN>
$flyersROV = ($current_year > 2008) ? 'Doc Ricketts' : 'Tiburon';
print "<BR><SPAN CLASS=flyerColor>Turquoise refers to the R/V <I>Western Flyer & $flyersROV</I></SPAN>";
$flyer, $tiburon, $wfly_starttime, $wfly_endtime, $wfly_url,
##print "<FONT SIZE=\"2\" COLOR=#009999>Flyer </FONT>" if $flyer;
##print "<IMG SRC=\"westernflyer-15.gif\"> " if $flyer;
print "<SPAN CLASS=flyerColor>";
print "$wfly_starttime" if $flyer && $wfly_starttime =~ /\d+:\d\d/;
print " - " if $flyer && ($wfly_starttime =~ /\d+:\d\d/ || $wfly_endtime =~ /\d+:\d\d/);
print "$wfly_endtime" if $flyer && $wfly_endtime =~ /\d+:\d\d/;
print " $flyer" if $flyer;
print " / " if $flyer && $tiburon;
$flyer, $tiburon, $wfly_starttime, $wfly_endtime, $wfly_url,
<TH BGCOLOR="#00F0F0">R/V <I>Western Flyer</I></TH>
<B>Chief Scientist / ship mission:</B> $flyer
$flyersROV = ($current_year > 2008) ? 'Doc Ricketts' : 'Tiburon';
print "<FONT SIZE=\"5\" COLOR=#009999 FACE=\"Technical\"><B><I>Western Flyer \/</I></B> </FONT>";
print "<FONT SIZE=\"5\" COLOR=#009999 FACE=\"Technical\"><B><I>$flyersROV: </I></B></FONT>";
print $query->textfield('FLYER',$defaults[11],80),"<BR>\n";
'PTLO_STARTTIME','PTLO_ENDTIME','FLYER',
$flyer_url = getEXPDinfo('wfly');
print "flyer_url = $flyer_url<BR>" if $debug;
$new_row .= $query->param('FLYER');
$new_row .= $flyer_url;
$get_expd_url = 'http://www.mbari.org/cruises/flyer/mission.asp';
➜ ~ ssh -Y webcalendar@pismo
webcalendar@pismo's password:
Last login: Mon Feb 27 16:23:23 2023 from mbari2333.shore.mbari.org
********************************************************
********************************************************
Welcome to the
Monterey Bay Aquarium Research Institute
All use must be in accordance with
MBARI's acceptable use policy
Unauthorized Access Prohibited
********************************************************
********************************************************
-sh-4.2$ cd dev/WebCalendar/scripts/
-sh-4.2$ grep -i flyer merge_zimbra.py
public_ship_folders = ['pf_rachel_carson', 'pf_westernflyer', 'pf_paragon']
'pf_westernflyer': 'flyer',
'pf_westernflyer': 'wfly',
16|11|2014|Teresa|Cardoza|wieteska@mbari.org||||||Robison||||http://www3.mbari.org/cruises/flyer/mission.asp?y=2014&d=320&dtg='11/16/14'&ship=wfly||||||1415036579
17|11|2014|Teresa|Cardoza|wieteska@mbari.org|Hobson||||http://www3.mbari.org/cruises/carson/mission.asp?y=2014&d=321&dtg='11/17/14'&ship=rcsn|Robison||||http://www3.mbari.org/cruises/flyer/mission.asp?y=2014&d=321&dtg='11/17/14'&ship=wfly||||||1415150939
Order is carson, flyer, Zephyr/Paragon with fields: summary|unused|unused|unused|expd_url
2. Assess the work to be done
The ship.list file format is rigid with the order of the carson, flyer, and paragon fields fixed at the end of the line. The fields are separated by the pipe character. As there is no overlap between the Flyer and the Packard schedules we can simply replace the flyer fields with the Packard fields. We should start by first modifying the merge_zimbra.py script to replace the flyer fields with the Packard fields. Then we can run the script and see what the results are. If the results are good we can then modify the calendar.cgi script to replace the flyer fields with the Packard fields. To test the operation of merge_zimbra.py we can execute it without the -o option, which will not overwrite the original ship.list file, e.g. for a short test period in July 2023 when the Packard is on its way to Moss Landing:
➜ ~ ssh -Y webcalendar@pismo
webcalendar@pismo's password:
Last login: Tue Feb 28 10:25:04 2023 from mbari2333.shore.mbari.org
********************************************************
********************************************************
Welcome to the
Monterey Bay Aquarium Research Institute
All use must be in accordance with
MBARI's acceptable use policy
Unauthorized Access Prohibited
********************************************************
********************************************************
-sh-4.2$ cd ~/dev/WebCalendar/scripts/
-sh-4.2$ source ~/VirtualEnvs/venv-calendar/bin/activate
(venv-calendar)-sh-4.2$ ./merge_zimbra.py --start 20230712T000000 --end 20230715T000000 -v 3
INFO 2023-02-28 10:49:16,350 merge_zimbra.py <module>:470 Collecting Zimbra ship calendar events between 2023-07-12 00:00:00 and 2023-07-15 00:00:00
INFO 2023-02-28 10:49:16,351 merge_zimbra.py <module>:474 Reading Zimbra calendar pf_rachel_carson
DEBUG 2023-02-28 10:49:20,036 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-14 00:00:00 Mob for pf_rachel_carson
DEBUG 2023-02-28 10:49:20,058 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-12 00:00:00 Durkin ROV for pf_rachel_carson
DEBUG 2023-02-28 10:49:20,066 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-13 00:00:00 Haddock ROV for pf_rachel_carson
INFO 2023-02-28 10:49:20,263 merge_zimbra.py <module>:474 Reading Zimbra calendar pf_westernflyer
INFO 2023-02-28 10:49:21,487 merge_zimbra.py <module>:474 Reading Zimbra calendar pf_paragon
DEBUG 2023-02-28 10:49:24,029 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-13 00:00:00 PI Chavez CS Wahl for pf_paragon
INFO 2023-02-28 10:49:24,159 merge_zimbra.py <module>:492 Formatting for web calendar using above settings...
INFO 2023-02-28 10:49:24,160 merge_zimbra.py <module>:495 3 items found from Zimbra ship calendars; merging with web calendar
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:371 12|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Durkin ROV|||||||||||||||1677610164
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:380 Web calendar and Zimbra agree for date: 2023-07-12 00:00:00
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:371 13|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Haddock ROV|||||||||||||||1677610164
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:393 Removing 1 items from the begining of lines_from_zimbra list
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:371 13|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Haddock ROV|||||||||||||||1677610164
INFO 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:377 Would overwrite with new zimbra_line for date: 2023-07-13 00:00:00
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:371 14|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Mob|||||||||||||||1677610164
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:393 Removing 1 items from the begining of lines_from_zimbra list
DEBUG 2023-02-28 10:49:24,202 merge_zimbra.py merge_lines:371 14|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Mob|||||||||||||||1677610164
DEBUG 2023-02-28 10:49:24,203 merge_zimbra.py merge_lines:380 Web calendar and Zimbra agree for date: 2023-07-14 00:00:00
DEBUG 2023-02-28 10:49:24,203 merge_zimbra.py merge_lines:385 No more lines_from_zimbra list
INFO 2023-02-28 10:49:24,205 merge_zimbra.py merge_lines:410 A total of 1 lines changed for web calendar ship.list file
There's not a lot to do. Basically, modifying at most a dozen lines or so, but we want to test carefully.
3. Test changes to merge_zimbra.py
Replace pf_westernflyer with pf_davidpackard (their names in Zimbra) in merge_zimbra.py:
(venv-calendar)-sh-4.2$ svn diff merge_zimbra.py
Index: merge_zimbra.py
===================================================================
--- merge_zimbra.py (revision 13914)
+++ merge_zimbra.py (working copy)
@@ -63,7 +63,7 @@
ch.setFormatter(formatter)
logger.addHandler(ch)
- public_ship_folders = ['pf_rachel_carson', 'pf_westernflyer', 'pf_paragon']
+ public_ship_folders = ['pf_rachel_carson', 'pf_davidpackard', 'pf_paragon']
def __init__(self):
'''
And test it:
(venv-calendar)-sh-4.2$ ./merge_zimbra.py --start 20230712T000000 --end 20230715T000000 -v 3
INFO 2023-02-28 10:55:44,113 merge_zimbra.py <module>:470 Collecting Zimbra ship calendar events between 2023-07-12 00:00:00 and 2023-07-15 00:00:00
INFO 2023-02-28 10:55:44,113 merge_zimbra.py <module>:474 Reading Zimbra calendar pf_rachel_carson
DEBUG 2023-02-28 10:55:48,057 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-14 00:00:00 Mob for pf_rachel_carson
DEBUG 2023-02-28 10:55:48,078 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-12 00:00:00 Durkin ROV for pf_rachel_carson
DEBUG 2023-02-28 10:55:48,086 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-13 00:00:00 Haddock ROV for pf_rachel_carson
INFO 2023-02-28 10:55:48,273 merge_zimbra.py <module>:474 Reading Zimbra calendar pf_davidpackard
DEBUG 2023-02-28 10:55:48,486 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-12 00:00:00 Vigo Local Ops for pf_davidpackard
DEBUG 2023-02-28 10:55:48,489 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-14 00:00:00 Vigo Local Ops for pf_davidpackard
DEBUG 2023-02-28 10:55:48,494 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-13 00:00:00 Vigo Local Ops for pf_davidpackard
INFO 2023-02-28 10:55:48,496 merge_zimbra.py <module>:474 Reading Zimbra calendar pf_paragon
DEBUG 2023-02-28 10:55:51,101 merge_zimbra.py read_zimbra_ship_calendar:126 Saving event for later formatting 2023-07-13 00:00:00 PI Chavez CS Wahl for pf_paragon
INFO 2023-02-28 10:55:51,222 merge_zimbra.py <module>:492 Formatting for web calendar using above settings...
INFO 2023-02-28 10:55:51,222 merge_zimbra.py <module>:495 3 items found from Zimbra ship calendars; merging with web calendar
DEBUG 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:371 12|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Durkin ROV|||||Vigo Local Ops||||||||||1677610551
INFO 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:377 Would overwrite with new zimbra_line for date: 2023-07-12 00:00:00
DEBUG 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:371 13|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Haddock ROV|||||Vigo Local Ops||||||||||1677610551
DEBUG 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:393 Removing 1 items from the begining of lines_from_zimbra list
DEBUG 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:371 13|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Haddock ROV|||||Vigo Local Ops||||||||||1677610551
INFO 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:377 Would overwrite with new zimbra_line for date: 2023-07-13 00:00:00
DEBUG 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:371 14|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Mob|||||Vigo Local Ops||||||||||1677610551
DEBUG 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:393 Removing 1 items from the begining of lines_from_zimbra list
DEBUG 2023-02-28 10:55:51,264 merge_zimbra.py merge_lines:371 14|07|2023|Zimbra|Calendar|pf_rachel_carson@mbari.org|Mob|||||Vigo Local Ops||||||||||1677610551
INFO 2023-02-28 10:55:51,265 merge_zimbra.py merge_lines:377 Would overwrite with new zimbra_line for date: 2023-07-14 00:00:00
DEBUG 2023-02-28 10:55:51,265 merge_zimbra.py merge_lines:385 No more lines_from_zimbra list
INFO 2023-02-28 10:55:51,267 merge_zimbra.py merge_lines:410 A total of 3 lines changed for web calendar ship.list file
Success! The Vigo Logo Ops events from Zimbra would be written to the ship.list file when run in production. :tada:
4. Test changes to calendar.cgi
The calendar.cgi script is the one that actually reads the ship.list file and displays the calendar. It is a Perl script that reads the ship.list file and formats HTML output. We can make changes to this file and test by reloading http://mww2.shore.mbari.org/events/calendar.cgi with the July 2023 month selected. Before making changes to calendar.cgi, we should make sure that the existing contents are in SVN version control. That is done by copying the file into the /u/webcalendar/dev/WebCalendar/internal/events directory on pismo and checking the differences:
(venv-calendar)-sh-4.2$ cp /var/www/html/internal/events/calendar.cgi ~/dev/WebCalendar/internal/events/
(venv-calendar)-sh-4.2$ svn diff ~/dev/WebCalendar/internal/events/calendar.cgi
Index: /u/webcalendar/dev/WebCalendar/internal/events/calendar.cgi
===================================================================
--- /u/webcalendar/dev/WebCalendar/internal/events/calendar.cgi (revision 13449)
+++ /u/webcalendar/dev/WebCalendar/internal/events/calendar.cgi (working copy)
@@ -37,7 +37,7 @@
$ship_lock_file = "$file_dir/ship_lock.file";
#$mailx="/usr/sbin/Mail ";
$mailx="/bin/mailx ";
- @pf_maillist=("mdalit\@mbari.org", "walsh\@mbari.org", "char\@mbari.org", "toni\@mbari.org");
+ @pf_maillist=("walsh\@mbari.org", "char\@mbari.org", "cassandra\@mbari.org", "mandy\@mbari.org", "krystle\@mbari.org", "gregv\@mbari.org");
@@ -310,7 +310,7 @@
<tr>
<td valign="top" width="20%"><a
href="http://www.mbari.org"><img
- src="http://www.mbari.org/images/mbarilogo-120_sh.gif" border="0" width="120"
+ src="http://www.mbari.org/wp-content/static/images/mbarilogo-120_sh.gif" border="0" width="120"
height="70"></a><br>
</td>
<td valign="middle" width="60%">
@@ -537,7 +537,7 @@
print "<P ALIGN=CENTER>(All times on this calendar are local)</P>\n";
print "<P ALIGN=\"CENTER\"><SPAN CLASS=underline>Abbreviations & color coding</SPAN>";
print "<BR><SPAN CLASS=carsonColor>Purple refers to the R/V <I>$ptlo_or_rcsn & Ventana</I></SPAN>";
- print "<BR><SPAN CLASS=flyerColor>Turqoise refers to the R/V <I>Western Flyer & $flyersROV</I></SPAN>";
+ print "<BR><SPAN CLASS=flyerColor>Turquoise refers to the R/V <I>Western Flyer & $flyersROV</I></SPAN>";
print "<BR><SPAN CLASS=zephyrColor>Blue refers to the R/V <I>Paragon</I></SPAN>";
print "<BR><SPAN CLASS=forumColor>Events in red take place in the Pacific Forum</SPAN>";
print "<BR>All other general events are in black";
@@ -545,9 +545,9 @@
# put in usage policy and application form
print "<P CLASS=\"smallerText\">";
print "Questions about events scheduled on this calendar may be \n";
- print "addressed to <A HREF=\"mailto:mdalit\@mbari.org\">Meilina Dalit</A>.\n";
+ print "addressed to <A HREF=\"mailto:krystle\@mbari.org\">Krystle Anderson</A>.\n";
print "<BR>Questions about ship scheduling may be \n";
- print "addressed to <A HREF=\"mailto:wieteska\@mbari.org\">Teresa Cardoza</A>.\n";
+ print "addressed to <A HREF=\"mailto:mschultz\@mbari.org\">Matt Schultz</A>.\n";
print "<BR>Questions about programming of this calendar application should ";
print "be addressed to <A HREF=\"mailto:mccann\@mbari.org\">Mike McCann</A>.\n";
print "</P>";
@@ -2327,9 +2327,9 @@
@defaults=($current_month,
$current_day,
$current_year,
- "Teresa",
- "Cardoza",
- "wieteska\@mbari.org",
+ "Matt",
+ "Schultz",
+ "mschultz\@mbari.org",
"",
"",
"",
Commit the changes to SVN:
(venv-calendar)-sh-4.2$ svn commit ~/dev/WebCalendar/internal/events/calendar.cgi -m "Update with new Logistics Coordinator and other changed contacts"
mccann@kahuna.shore.mbari.org's password:
Sending calendar.cgi
Transmitting file data .
Committed revision 13915.
Note that the changes we make will affect the naming/labeling of all events in the calendar. For example, past R/V Western Flyer events will be labeled R/V David Packard events.
Edit the calendar.cgi file in the ~/dev/WebCalendar/internal/events directory and make the following changes:
(venv-calendar)-sh-4.2$ diff -u ~/dev/WebCalendar/internal/events/calendar.cgi /var/www/html/internal/events/calendar.cgi
--- /u/webcalendar/dev/WebCalendar/internal/events/calendar.cgi 2023-02-28 11:22:42.432767000 -0800
+++ /var/www/html/internal/events/calendar.cgi 2023-02-28 11:50:40.156601121 -0800
@@ -320,7 +320,7 @@
<td valign="top" width="20%">
<P CLASS=smallerText>
<SPAN CLASS=carsonColor>R/V <I>$ptlo_or_rcsn & Ventana</I></SPAN>
- <BR><SPAN CLASS=flyerColor>R/V <I>Western Flyer & $flyersROV</I></SPAN>
+ <BR><SPAN CLASS=flyerColor>R/V <I>David Packard & $flyersROV</I></SPAN>
<BR><SPAN CLASS=zephyrColor>R/V <I>Paragon</I></SPAN>
<BR><SPAN CLASS=forumColor>Pacific Forum</SPAN>
<BR><SPAN CLASS=generalColor>General event</SPAN>
@@ -537,7 +537,7 @@
print "<P ALIGN=CENTER>(All times on this calendar are local)</P>\n";
print "<P ALIGN=\"CENTER\"><SPAN CLASS=underline>Abbreviations & color coding</SPAN>";
print "<BR><SPAN CLASS=carsonColor>Purple refers to the R/V <I>$ptlo_or_rcsn & Ventana</I></SPAN>";
- print "<BR><SPAN CLASS=flyerColor>Turquoise refers to the R/V <I>Western Flyer & $flyersROV</I></SPAN>";
+ print "<BR><SPAN CLASS=flyerColor>Turquoise refers to the R/V <I>David Packard & $flyersROV</I></SPAN>";
print "<BR><SPAN CLASS=zephyrColor>Blue refers to the R/V <I>Paragon</I></SPAN>";
print "<BR><SPAN CLASS=forumColor>Events in red take place in the Pacific Forum</SPAN>";
print "<BR>All other general events are in black";
@@ -1196,7 +1196,7 @@
print "<TH BGCOLOR=\"#00F0F0\">R/V <I>Rachel Carson</I></TH>\n";
}
print <<" end_of_html";
- <TH BGCOLOR="#00F0F0">R/V <I>Western Flyer</I></TH>
+ <TH BGCOLOR="#00F0F0">R/V <I>David Packard</I></TH>
<TH BGCOLOR="#00F0F0">R/V <I>Paragon</I></TH>
</TR>
<TR>
@@ -2391,7 +2391,7 @@
print "<P><HR WIDTH=50%><P>\n";
$flyersROV = ($current_year > 2008) ? 'Doc Ricketts' : 'Tiburon';
- print "<FONT SIZE=\"5\" COLOR=#009999 FACE=\"Technical\"><B><I>Western Flyer \/</I></B> </FONT>";
+ print "<FONT SIZE=\"5\" COLOR=#009999 FACE=\"Technical\"><B><I>David Packard \/</I></B> </FONT>";
print "<FONT SIZE=\"5\" COLOR=#009999 FACE=\"Technical\"><B><I>$flyersROV: </I></B></FONT>";
print "</CENTER>\n";
print "<B><FONT SIZE=\"3\">Chief Scientist:</FONT></B>\n<BR>\n";
It may be confusing for future maintainers, but there was no need to change the $flyer and $flyersROV variable names. There are other references in the calendar.css file that would also need to be changed. These are simply internal variable names that have no effect on the calendar's appearance, which looks good: http://mww2.shore.mbari.org/events/calendar.cgi :tada:. Commit the changes to SVN:
(venv-calendar)-sh-4.2$ cp /var/www/html/internal/events/calendar.cgi ~/dev/WebCalendar/internal/events/
(venv-calendar)-sh-4.2$ svn commit ~/dev/WebCalendar/internal/events/calendar.cgi -m "Replace R/V Western Flyer labels with R/V David Packard labels"
mccann@kahuna.shore.mbari.org's password:
Sending calendar.cgi
Transmitting file data .
Committed revision 13916.
4. Commit the change of merge_zimbra.py to SVN and test the cron job
Replace 'pf_westernflyer' with 'pf_davidpackard':
(venv-calendar)-sh-4.2$ svn diff merge_zimbra.py
Index: merge_zimbra.py
===================================================================
--- merge_zimbra.py (revision 13914)
+++ merge_zimbra.py (working copy)
@@ -63,7 +63,7 @@
ch.setFormatter(formatter)
logger.addHandler(ch)
- public_ship_folders = ['pf_rachel_carson', 'pf_westernflyer', 'pf_paragon']
+ public_ship_folders = ['pf_rachel_carson', 'pf_davidpackard', 'pf_paragon']
def __init__(self):
'''
(venv-calendar)-sh-4.2$ svn commit merge_zimbra.py -m "Replace 'pf_westernflyer' with 'pf_davidpackard'"
mccann@kahuna.shore.mbari.org's password:
Sending merge_zimbra.py
Transmitting file data .
Committed revision 13917.
Test the cron job:
(venv-calendar)-sh-4.2$ crontab -l
# Keep MBARI Web Calendar in sync with Zimbra ship schedules and Expd pre-cruise links
##0 * * * * dev/WebCalendar/scripts/merge_zimbra_hourly.sh >> logs/cron.log 2>&1
30 7,12,16,21 * * * dev/WebCalendar/scripts/merge_zimbra_expd.sh >> logs/cron.log 2>&1
(venv-calendar)-sh-4.2$ dev/WebCalendar/scripts/merge_zimbra_expd.sh
-sh: dev/WebCalendar/scripts/merge_zimbra_expd.sh: No such file or directory
(venv-calendar)-sh-4.2$ cd
(venv-calendar)-sh-4.2$ dev/WebCalendar/scripts/merge_zimbra_expd.sh
DEBUG 2023-02-28 12:08:23,251 merge_zimbra.py configure_file_logger:90 Configuring logging to go to file /u/webcalendar/logs/merge_zimbra_20230228.log
Traceback (most recent call last):
File "./merge_zimbra.py", line 476, in <module>
z.read_zimbra_ship_calendar(ship, after_date, before_date)
File "./merge_zimbra.py", line 120, in read_zimbra_ship_calendar
if parser.parse(self.args.start) < dtstart and dtstart < parser.parse(self.args.start):
File "/u/webcalendar/VirtualEnvs/venv-calendar/lib/python2.7/site-packages/dateutil/parser.py", line 1168, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/u/webcalendar/VirtualEnvs/venv-calendar/lib/python2.7/site-packages/dateutil/parser.py", line 556, in parse
res, skipped_tokens = self._parse(timestr, **kwargs)
File "/u/webcalendar/VirtualEnvs/venv-calendar/lib/python2.7/site-packages/dateutil/parser.py", line 675, in _parse
l = _timelex.split(timestr) # Splits the timestr into tokens
File "/u/webcalendar/VirtualEnvs/venv-calendar/lib/python2.7/site-packages/dateutil/parser.py", line 192, in split
return list(cls(s))
File "/u/webcalendar/VirtualEnvs/venv-calendar/lib/python2.7/site-packages/dateutil/parser.py", line 61, in __init__
'{itype}'.format(itype=instream.__class__.__name__))
TypeError: Parser must be a string or character stream, not NoneType
Oops! Looks like we need to test for the --start and --end arguments before using them, let's ask for forgiveness instead of permission:
-sh-4.2$ svn diff merge_zimbra.py
Index: merge_zimbra.py
===================================================================
--- merge_zimbra.py (revision 13917)
+++ merge_zimbra.py (working copy)
@@ -117,7 +117,10 @@
# Convert all dates to datetime at midnight so that they can be compared
dtstart = datetime.combine(c.decoded('dtstart'), datetime.min.time())
dtend = datetime.combine(c.decoded('dtend'), datetime.min.time())
- if parser.parse(self.args.start) < dtstart and dtstart < parser.parse(self.args.start):
+ try:
+ if parser.parse(self.args.start) < dtstart and dtstart < parser.parse(self.args.start):
+ self.logger.debug('dtstart = %s', dtstart)
+ except TypeError:
self.logger.debug('dtstart = %s', dtstart)
if dtstart >= after_date:
if before_date:
More testing:
(venv-calendar)-sh-4.2$ dev/WebCalendar/scripts/merge_zimbra_expd.sh
DEBUG 2023-02-28 12:14:16,985 merge_zimbra.py configure_file_logger:90 Configuring logging to go to file /u/webcalendar/logs/merge_zimbra_20230228.log
Traceback (most recent call last):
File "./merge_zimbra.py", line 496, in <module>
lines_from_zimbra = z.make_webcal_lines(OrderedDict(sorted(z.webcal_info.items())), check_for_precuise=cfp, update_zimbra_with_url=uzwu)
File "./merge_zimbra.py", line 196, in make_webcal_lines
cs_ln, expd_url = self.get_expd_url(dt, ship)
File "./merge_zimbra.py", line 157, in get_expd_url
expd_url = 'http://www3.mbari.org/cruises/%s/mission.asp?' % ship_lookup[ship]
KeyError: 'pf_davidpackard'
More fixes:
(venv-calendar)-sh-4.2$ svn diff merge_zimbra.py
Index: merge_zimbra.py
===================================================================
--- merge_zimbra.py (revision 13917)
+++ merge_zimbra.py (working copy)
@@ -117,7 +117,10 @@
# Convert all dates to datetime at midnight so that they can be compared
dtstart = datetime.combine(c.decoded('dtstart'), datetime.min.time())
dtend = datetime.combine(c.decoded('dtend'), datetime.min.time())
- if parser.parse(self.args.start) < dtstart and dtstart < parser.parse(self.args.start):
+ try:
+ if parser.parse(self.args.start) < dtstart and dtstart < parser.parse(self.args.start):
+ self.logger.debug('dtstart = %s', dtstart)
+ except TypeError:
self.logger.debug('dtstart = %s', dtstart)
if dtstart >= after_date:
if before_date:
@@ -144,11 +147,11 @@
(Chief Scientist last name, expd url) if the precruise exists
'''
ship_lookup = {'pf_rachel_carson': 'carson',
- 'pf_westernflyer': 'flyer',
+ 'pf_davidpackard': 'packard',
'pf_paragon': 'paragon',
}
ship_code_lookup = {'pf_rachel_carson': 'rcsn',
- 'pf_westernflyer': 'wfly',
+ 'pf_davidpackard': 'dpkd',
'pf_paragon': 'prgn',
}
expd_url = 'http://www3.mbari.org/cruises/%s/mission.asp?' % ship_lookup[ship]
More testing:
(venv-calendar)-sh-4.2$ ~/dev/WebCalendar/scripts/merge_zimbra_expd.sh
DEBUG 2023-02-28 12:22:07,966 merge_zimbra.py configure_file_logger:90 Configuring logging to go to file /u/webcalendar/logs/merge_zimbra_20230228.log
Traceback (most recent call last):
File "./merge_zimbra.py", line 496, in <module>
lines_from_zimbra = z.make_webcal_lines(OrderedDict(sorted(z.webcal_info.items())), check_for_precuise=cfp, update_zimbra_with_url=uzwu)
File "./merge_zimbra.py", line 196, in make_webcal_lines
cs_ln, expd_url = self.get_expd_url(dt, ship)
File "./merge_zimbra.py", line 165, in get_expd_url
contents = response.read()
UnboundLocalError: local variable 'response' referenced before assignment
This is likely due to the expd_url not being implemented for http://www3.mbari.org/cruises/packard/mission.asp?. www3 is the IIS server sirocco. This problem is fixed by mounting cifs://sirocco.shore.mbari.org/wwwroot and copying the contents of the cruises/flyer directory to cruises/packard and making appropriate changes, namely changing 'Western Flyer' to 'David Packard' in cruises/packard/mission.asp. There are other wf spcific files in the cruises/packard directory that can be fixed later as needed. For now we want the calendar to work. Test again:
(venv-calendar)-sh-4.2$ cd
(venv-calendar)-sh-4.2$ ~/dev/WebCalendar/scripts/merge_zimbra_expd.sh
DEBUG 2023-02-28 12:38:18,415 merge_zimbra.py configure_file_logger:90 Configuring logging to go to file /u/webcalendar/logs/merge_zimbra_20230228.log
No errors! Let's see what the output looks like for July 2023:
Looks Good! :tada:
5. Make sure we have committed all our changes
(venv-calendar)-sh-4.2$ cd ~/dev/WebCalendar/scripts/
(venv-calendar)-sh-4.2$ svn diff merge_zimbra.py
Index: merge_zimbra.py
===================================================================
--- merge_zimbra.py (revision 13917)
+++ merge_zimbra.py (working copy)
@@ -117,7 +117,10 @@
# Convert all dates to datetime at midnight so that they can be compared
dtstart = datetime.combine(c.decoded('dtstart'), datetime.min.time())
dtend = datetime.combine(c.decoded('dtend'), datetime.min.time())
- if parser.parse(self.args.start) < dtstart and dtstart < parser.parse(self.args.start):
+ try:
+ if parser.parse(self.args.start) < dtstart and dtstart < parser.parse(self.args.start):
+ self.logger.debug('dtstart = %s', dtstart)
+ except TypeError:
self.logger.debug('dtstart = %s', dtstart)
if dtstart >= after_date:
if before_date:
@@ -144,11 +147,11 @@
(Chief Scientist last name, expd url) if the precruise exists
'''
ship_lookup = {'pf_rachel_carson': 'carson',
- 'pf_westernflyer': 'flyer',
+ 'pf_davidpackard': 'packard',
'pf_paragon': 'paragon',
}
ship_code_lookup = {'pf_rachel_carson': 'rcsn',
- 'pf_westernflyer': 'wfly',
+ 'pf_davidpackard': 'dpkd',
'pf_paragon': 'prgn',
}
expd_url = 'http://www3.mbari.org/cruises/%s/mission.asp?' % ship_lookup[ship]
(venv-calendar)-sh-4.2$ svn commit merge_zimbra.py -m "Fixes following change from Western Flyer to David Packard"
mccann@kahuna.shore.mbari.org's password:
mccann@kahuna.shore.mbari.org's password:
mccann@kahuna.shore.mbari.org's password:
Sending merge_zimbra.py
Transmitting file data .
Committed revision 13918.
Now we monitor for changes in the Zimbra calendar properly migrating to the internal web calendar.