This series is written by a representative of the latter group, which is comprised mostly of what might be called "productivity users" (perhaps "tinkerly productivity users?"). Though my lack of training precludes me from writing code or improving anyone else's, I can, nonetheless, try and figure out creative ways of utilizing open source programs. And again, because of my lack of expertise, though I may be capable of deploying open source programs in creative ways, my modest technical acumen hinders me from utilizing those programs in what may be the most optimal ways. The open-source character, then, of this series, consists in my presentation to the community of open source users and programmers of my own crude and halting attempts at accomplishing computing tasks, in the hope that those who are more knowledgeable than me can offer advice, alternatives, and corrections. The desired end result is the discovery, through a communal process, of optimal and/or alternate ways of accomplishing the sorts of tasks that I and other open source productivity users need to perform.

Thursday, February 4, 2016

Miscellaneous Thursday quickies: what's your bi-directional syncing utility?

So I've been pursuing a research project for the last year or so and have been locating and saving material related to it, as well as doing some of my own writing in the area. I keep that material in a particular folder. That's all fine and good. The problem is that I want the ability to work on the project while I'm at any of 3 different computers--computers that are often located in 3 different locales, some of which are even remote from my LAN. So, how to host the same folder on all three machines, and keep it current with the most recent changes made on any of the 3 computers?

I intend for this to be a manual process, i.e., one that will involve me manually running some program or script on each of the three machines, in order to update the folder. I should also mention that I have access to a shell account where I can run a number of utilities that can facilitate this--so a 4th computer, technically speaking, is involved as well. I envision the shell account functioning as a sort of central hub for keeping said folders in sync: a sort of master copy of the folder can be stored there and each of the three machines can syncronize with that folder as need will arise.

I'm still trying to puzzle out how to pull all this together and am looking at the sorts of software/utilities that can accomplish the task. I've only tested out one option thus far--bsync. I opted for that in an initial foray for its simplicity: it's just a python script that enhances the functionality of rsync (a great sync utility, but one that does not do bi-directional synchronization). So all I needed to do was download the script and make it executable.

Using the utility, I was able to put the most current copy of the folder at my shell account by just running bsync MyFolder me@my.shellacct.com:MyFolder (the MyFolder directory must already exist at the remote address). So I've at least made a beginning.

That said, I'm still in the early stages of investigating approaches to do the sort of bi-directional synchronization I'm after. Tests with bsync have gone well so far but, if I'm understanding correctly, this utility does not deal well with sub-folders--which could be an issue in my use scenario; it seems bsync will work best on a folder or directory that contains only files, while my directory has a few sub-directories under it.

Other possible options I've found are csync (which uses smb or sftp), osync, bitpocket, and FreeFileSync. The first 3 of these are most attractive to me since they are command-line utilities. FreeFileSync is a graphical utility, though it does appear that it can be run from the command line as well. I should also mention unison, which I've looked at but not pursued--the reason being that it apparently requires that the same version be installed on all concerned machines, which is something that will be unrealistic in my case (Arch runs on 2 machines, an older Ubuntu on another, and BSD on the fourth).

So, what is your bi-directional synchronization software preference? Any further tips or pointers to add on accomplishing this task?

Tuesday, January 12, 2016

Addendum to 11th installment: Lynx; scraping credentialed web pages

Sort of a dramatized headline for what I've accomplished using the command-line Lynx  browser, but not too far from the mark. I've described in previous entries how I've used lynx to accomplish similar goals of extracting target information from web pages, so this entry is a continuation along those same lines.

I recently signed up for a prepaid cellular plan touted as being free, though it is one limited to a certain (unreasonably low, for most) number of minutes per month. The plan has thus far worked well for me. The only real issue I have come across is that I had not yet discovered any way easily to check how many minutes I've used and how many are left. The company providing the service is, of course, not very forthcoming with that sort of information: they have a vested interest in getting you to use up your free minutes, hoping thereby that you'll realize you should buy a paid plan from them, one that includes more minutes. The only way I'd found for checking current usage status is to log in to their web site and click around til you reach a page showing that data.

Of course I am generally aware of the phenomemon of web-page scraping and also have heard of python and/or perl scripts that can perform more or less automated interactions with web pages (youtube-dl being one example). So I initally thought my task would require something along these lines--quite the tall order for someone such as myself, knowing next to nothing about programming in either python or perl. But then I ran across promising information that led me to believe I might well be able to accomplish this task using the tried and true lynx browser, and some experimentation proved that this would, indeed, allow me to realize my goal.

The information I discovered came from this page. There is found a description of how it is possible to record to a log file all keystrokes entered into a particular lynx browsing session--something reminiscent of the way I used to create macros under Microsoft Word when I was using that software years ago. The generated log file can then, in turn, be fed to a subsequent lynx session, effectively automating certain browsing tasks, such as logging into a site, navigating to, then printing (to a file, in my case) a page. Add a few other utilities like cron, sed, and mail, and I have a good recipe for getting the cellular information I need into an e-mail that gets delivered to my inbox on a regular basis.

The initial step was to create the log file. An example of the command issued is as follows:

lynx -cmd_log=/tmp/mysite.txt http://www.mysite.com.

That, of course, opens the URL specified in lynx. The next step is to enter such keystrokes are are necessary to get to the target page. In my case, I needed to press the down arrow key a few times to reach the login and password entry blanks. I then typed in the credentials, hit the down arrow again, then the "enter" key to submit the credentials. I then needed to hit the "end" key on the next page, which took me all the way to the bottom of that page, then the up arrow key a couple of times to get to the link leading to the target page. Once I got to the target page, I pressed the "p" key (for print), then the "enter" key (for print to file), at which point I was prompted for a file name. Once I'd entered the desired file name and pressed the "enter" key again, I hit the "q" key to exit lynx. In this way, I produced the log file I could then use for a future automated session at that same site. Subsequent testing using the command

lynx -cmd_script=mysite.txt http://www.mysite.com

confirmed that I had, in fact, a working log file that could be used for retreiving the desired content from the target page.

The additional steps for my scenario were to turn this into a cron job (no systemd silliness here!), use sed to strip out extraneous content from the beginning and end of the page I'd printed/retrieved, and to get the resulting material into the body of an e-mail that I would have sent to myself at given intervals. The sed/mail part of this goes something like

sed -n 24,32p filename | mail -s prepaid-status me@mymail.com*

* I can't go into particulars of the mail program here, but suffice to say at least that you need a properly edited configuration file for your mail sending utility (I use msmtp) for this to work.

Friday, April 17, 2015

Addendum to 12th installment: watermarks with copyright notice using LaTeX

So, it's been awhile. And there's been plenty I could have blogged about on the tech front. Like when I copied my Arch install to another hard drive, making it bootable. But I didn't. And now I've forgotten important details of how I did it. Oh well.

I can blog about this next item, though, which is still fresh in memory. I've got to write up  some articles and am sending them out for proofreading. So I wanted to mark them as drafts, something I already know how to do and have blogged about previously.

I decided to modify things a bit for the current task, though. This time I'm using only one utility to do the watermarking--LaTeX--and I'm tweaking things a bit further.

The challenge this time is making a watermark with a line break, as well as one that contains text with differing font sizes in the two lines. I want a really large font for the first line, which marks the document as a draft--as in my previous installment--but I want a really small font for the second line this time. That second line is where a copyright notice will be located.

Without further ado, here's the MWE (TeX-speak for minimum working example) I've come up with for accomplishing this:



This will place, diagonally across each page of the document, a notice in light gray font and in very large letters, with the word DRAFT. Underneath that, there will be text in much smaller font alerting that the material is under copyright claim of the document's author. It's also got a nice little feature that auto-inserts the year, so it's something that can be reused in varying documents over a period of time, relieving the composer of having to fiddle with minor details like dates.

So, that's about it for this installment!

LATE ADDITION: Just today I ran across a new means of watermarking that can be done on already-existing .pdf files. It involves using the program pdftk and is quite simple. You simply create an empty--except for your desired watermark--.pdf, then use the program to add the watermark to the already-existing .pdf. Something like the following:

pdftk in.pdf background back.pdf output out.pdf

(I ran across that here). I used LibreOffice Draw to create such a background watermark and easily added that to an existing .pdf. It worked great, though it should be noted that the watermark won't cover graphics; I assume there must be a way to make it do so, however.

Tuesday, November 18, 2014

Miscellaneous Tuesday quickies: creating and using ssh tunnels/proxies

This entry will concern tunneling so as to get around port-blocking restrictions. It's something I learned about some years ago, but had a difficult time wrapping my head around it. While I can't say I understand it a whole lot better now, I can at least say that I've been able to get it working.

In my case it was needed because I've been working in a library whose wifi network is set up to block a variety of non-standard ports. That's a problem for me since I run my (command-line) e-mail client on a home computer, and I connect to that computer via ssh--which, in turn, runs on a non-standard port. So, when I work in this library, I am unable to connect to my home machine to do e-mailing. There are also occasional problems with sites being blocked on this network (and, no, I'm not rying to view porn).

For this to work, one must have access to some machine outside the wifi network that runs ssh on port 443. I happen to have a shell account with a service that has just such a set-up.

In my case, then, I can set up the tunnel as follows:

ssh -L localhost:1234:my.dyndns.url:12345 -p 443 my-user@my.shellacct.net.

I am asked for my password, then logged into my shell account, and the tunnel is thus opened.

Then, to connect to ssh as running on my home machine, I simply issue

ssh -p 1234

To get around the occasional page blocking I've run into, I first downloaded a browser I will dedicate to this task--namely, qupzilla. Then, I need to set up a socks proxy, which is done via ssh, like so:

ssh -D 8080 my-user@my.shellacct.net -p 443

After that, it's a matter of configuring qupzilla (or your preferred browser) to route web traffic over the socks proxy you've just created. That's done by going to Edit > Preferences > Proxy Configuration, ticking the Manual configuration radio button, selecting socks5 from the drop-down menu, then entering localhost into the first field next to that and 8080 in the Port field. Click Apply and Ok, and qupzilla will be set to route its traffic over your proxy, thus avoiding the blocks instituted by the wifi network.

With this basic information, it should be clear how other sorts of ssh tunnels and/or proxies could be set up.

Friday, July 18, 2014

Miscellaneous Friday quickies: The Plop boot manager; what is it and why would you need it?

Prefatory remark: I am uncertain of the licensing status of the project discussed in the posting below, but I suspect it may not--unlike most of the other utilities I discuss in this blog--be open-source.

Unless you, like me, are stubbornly trying to repurpose aging hardware, this tool might not be of much interest to you. But it allowed me to get an older machine booting from USB when BIOS limitations were interfering, , so I decided to document here the fairly simple procedures I followed to accomplish this in case they might be of benefit to others.

How old was said machine? Well, old enough to not only have problems booting from USB flash drives (BIOS USB boot options were limited to USB floppies or ZIP disks), but to have a floppy drive in it as well! A single core machine, as you might guess, although the motherboard did at least have SATA headers--which made it a good candidate for the project I had in mind.

I learned, through some on-line research, about the Plop boot manager--touted for enabling systems to boot from USB even where BIOS settings limited it--and that floppy disk images of the boot manager are included in the download. So I dug up and dusted off a floppy, downloaded the image, and wrote it to the floppy the GNU/Linux way--using dd:

dd if=/path/to/plpbt.img of=/dev/fd0

And that disk did, in fact, allow me to boot sanely from a USB thumb drive I'd plugged into the system. On boot, a starfield simulation reminiscent of the old Star Trek intro (ok, I'm dating myself here) appeared on the screen, in the foreground of which was a boot menu from which I could select the medium I wished to boot. And, sure enough, USB was one of the items.

That wasn't quite all I needed for my own application, however; you see, my hope was to have this machine run headless. So, how to make the boot manager default to booting from the USB drive after a certain number of seconds?

For that, it turns out, I needed another program included in the download called plpbtcfg. That program is what allows one to modify the binary file plpbt.bin. And plpbt.bin needs to be accessed somehow as well in order to modify it--accomplished in my case by mounting plpbt.img as a looped file system.

So I ran mount -o loop /path/to/plpbt.img /mnt/loop. Once that image had been thus mounted, I cd'd to where I'd downloaded plpbtcfg and ran plpcfgbt cnt=on cntval=4 dbt=usb /mnt/loop/plpbt.bin: that gave the boot menu a four-second time count, after which the computer would automatically boot from USB. I rewrote, using dd again, that image, to the floppy. So, mission accomplished.

Except some other aspects of that machine's operation proved not very suitable to the application I was hoping to deploy it for, so I'm not sure it will finally be put into service. But that's another story . . .

Saturday, April 12, 2014

Miscellaneous Friday quickies: crop pdf margins with psnup

As sometimes happens, I recently needed to print off a portion of a public-domain work that's been scanned to portable document format and made available for download via Google Books. As the case proves to be at times, the original book had pages with fairly wide margins; when that sort of scanned book gets printed on letter-sized paper, you end up with a smallish text-box in the middle of a page that will have something like two-inch margins. That makes the text harder to read because the font ends up being relatively small, and it also results in waste of a lot of paper.

What to do, then, to make the text larger and make it occupy more of the page? I used three utilities for this job: xpdf to excise the target pages into a postscript file, psnup to enlarge the text and crop margins, and ps2pdf to convert the modified pages back to a pdf file. psnup is part of the psutils package, while ps2pdf relies on Ghostscript. A description of the steps I took follows.

With the pdf-viewer xpdf, the print dialog offer two option: print the document to a physical printer, or print to a postscript file. Both options allow for page-range stipulation. That's how I created a postscript file from the target pages.

Next, psnup--a really handy tool I've used previously for creating booklets (more about that in a future entry), but which I'd never considered might perform a job like this--was used to reduce margins, which had the added effect of increasing the text size. The command I used, which I appropriated from here, looks like this:

psnup -1 -b-200 infile.ps file_zoomed.ps

The crucial switch here seems to be -b (which is short for borders) followed by a negative numeral. Of course the numeral 200 as seen in this example will need to be modified to suit your circumstances.

The final step of converting file_zoomed.ps, using ps2pdf, to a pdf was simplest of all--in fact so simple that I won't even describe it here. I hope this brief description may be an aid to others wishing to execute a task like this.

Finally, I ran across some other related information of interest while reaserching how to crop pdf margins. Here, for example, is a page that describes a few other, more technically-involved ways to reduce margins: http://ma.juii.net/blog/scale-page-content-of-pdf-files. This posting on the Ubuntu forums has a really interesting script someone put together for making pdf booklets: http://ubuntuforums.org/showthread.php?t=1148764. And this one offers some clever tips on using another psutils utility--psbook--for making booklets: http://www.peppertop.com/blog/?p=35. Then, there's what might be the most straightforward utility of all for cropping pdf margins--pdfcrop. Since I could not get it to work for me in this instance, I looked for and found the alternative described above.

Monday, March 17, 2014

Miscellaneous Monday quickies: volume adjustment via keyboard

Being an enthusiast of minimalist GUI systems, I'd heard some time ago of the i3 window manager and liked what I'd read. Recently, I switched over a couple of my computers to it and have been quite happy with it.

I ran across a news item the other day that was touting the virtues of i3 and which therefore caught my interest. Especially intriguing was the author's description of how, using that WM, certain keyboard keys or key combinations could be mapped so as to govern the computer's sound ouput--intriguing, that is, even apart from the fact that it was a description of a system configured to use pulse audio for sound output (my preference for ALSA over pulse is material for another entry). Still, I felt it should not be too hard to modify those directions to suit my systems.

As in the article referenced, it was a simple matter of modifying ~/.i3/config, adding some lines. In my case, the lines were as follows:



The keyboard on that particular machine is what I believe is called a "multimedia keyboard," meaning that it has a few keys dedicated, rather than to alphanumeric characters, to multimedia functions such as volume control. Finding which key codes to place in that file was a simple matter of using the xev utility. The ALSA--as opposed to pulse audio-- commands for raising, lowering, and muting volume were readily found in an internet search.

After a few trial runs and further tweaks, a quick restart of i3 revealed that things were working as expected. Flush with success from that project, I decided I might get the same thing working on another computer in my apartment--though that computer, since it needs to be usable for my wife, runs a different WM; incidentally, it runs JWM with a home-brewed Gnome 2 mock-up interface (I plan to do a write-up someday describing the Gnome 2 mock-up I created).

The basic idea of getting keyboard keys controlling volume is the same, though it involves editing a different configuration file--named ~/.jwmrc--that uses alternate syntax. Since the keyboard attched to this machine is not a multimedia keybaord, I ended up repurposing some seldom-used keys, in combination with the Alt key, for volume control functions. The entries in that file are as follows:



That pretty much sums it up this quickie entry.