Working with web application with version control

I have a web application to put into version control.

I decided not to add my NetBeans project to version control.

I created a folder that contained all the files for the web application.

I created a new NetBeans project from existing sources. I pointed all source files to the version controlled sources. The sources are not in the NetBeans project folder.

From Window->Favorites I selected the version controlled folder. From Team->Subversion, I imported the version controlled folder.

It seems that NetBeans is smart about what should go into version control. I probable could have avoided this altogether!

Sharing a repository with SVN and GIT

It is always difficult to get something to work when you don't understand the software. I had this problem with Git. I was using it, but did not understand it. Then I read the first four chapters of the Git book.

I wanted to import a repository that was built in Git into Svn. The project was originally a Svn project, but new developers ported it to Git. I wanted to be able to modify it from either Git or Svn. I knew that I would not be able to create a seamless integration with the first Svn version, but I would be happy with a new one that could work with both from now on.

I found an excellent reference on Stack Overflow. Of course, it did not work with my repository. After many hours, I got it to work.

From Stack Overflow:

1. cd /path/to/git/localrepo
2. svn mkdir --parents protocol:///path/to/repo/PROJECT/trunk -m "Importing git repo"
3. git svn init protocol:///path/to/repo/PROJECT -s
4. git svn fetch
5. git rebase trunk
5.1.  git status
5.2.  git add (conflicted-files)
5.3.  git rebase --continue
5.4.  (repeat 5.1.)
6. git svn dcommit

When I reached the git rebase trunk line, I was faced with so many conflicts, it was impossible to continue. I still do not understand why the replay of the previous commits was causing so many (add/add) conflicts.

I then tried another sequence from later in the post:

git svn fetch

git rev-list --parents master | grep '^.\{40\}$'

git rev-parse svn/trunk

echo   >> .git/info/grafts

git filter-branch -- ^svn/trunk --all

rm .git/info/grafts

More voodoo. In this sequence, I was introduced to rev-list, rev-parse and filter-branch. I am still a little vague on how these work. I was able to get this to work, and was introduced to a nice GUI for a git repository, gitk.

The next statement was git svn rebase. Again, too many (add/add) conflicts.

I then learned that git rebase has the -s for defining a strategy. With the recursive strategy, it is possible to set an option for always accepting the new revision into the current one.

git rebase -s recursive -Xtheirs svn/trunk

This worked, but then I received an error about a commit that did not have a message. Apparently, some commits were added without messages and this was preventing the rebase.

I found another stack overflow post that explained how to add messages for commits that are missing messages. The technique I picked only works for single line messages:

git filter-branch -f --msg-filter '
read msg
if [ -n "$msg" ] ; then
    echo "$msg"
else
    echo "The commit message was empty"
fi' 

I then aborted the rebase and reran it. I received a bad format error. Filter-branch rehashes all the data, so I had to restart the sequence again, starting with git svn fetch.

Once that was done, I was able to complete the sequence:

git rebase -s recursive -Xtheirs svn/trunk
git svn dcommit

It worked. I can now do as the post suggested:

You can now sync from svn to git, using the following commands:

git svn fetch
git rebase trunk

And to sync from git to svn, use:

git svn dcommit

Output of cal on Linux contains underscore and backspace

I just ran across some strange behavior in the output of the cal command on Linux.

I am running a perl script on Linux. The script retrieves the output of the Linux cal command with

`cal $month $year`

where $month and $year are set in the script.

I have been running this script for years and have not had a problem before.

The output of the calendar for

cal 5 2014

contains some control characters. For day 18, the output of the day is _1_8, where is the backspace key (hex 08). As you can guess, this runs havoc with my script that is formatting the calendar.

I had to filter the output of the command, replacing occurrences of _

s/_\x08//g

After filtering the output, the script ran as expected.

I ran a test script on all calendars from 1800 to 3000. Only May 2014 has the error.

Moving a Firefox extension from Addon-builder to local SDK

A couple of years ago, I developed a Firefox addon using the Addon Builder. The Addon Builder is no longer available. I am developing on a Mac.

I followed the steps at https://developer.mozilla.org/en-US/Add-ons/SDK/Tutorials/Installation to create a new addon using the locally installed SDK.

Open a terminal window and initialize the SDK. Change to the install directory for the SDK and issue the command source bin/activate.

Create a new directory for the root of the extension. Change to the directory and issue cfx init. The name of the extension is the name of the root folder.

Next, I downloaded my old extension from the builder dashboard. (Update: I can no longer find the builder dashboard, so I accessed my profile from the addons listing for me: https://addons.mozilla.org/en-us/firefox/user/username/, replacing username with my user name. Then, instead of clicking Add To Firefox, select Save As from the context menu of the button.) I copied my lib/main.js file and the files in the data folder into the extension I just created with the SDK. I do not know if I could have just edited the unzipped XPI file that I downloaded.

Change to the root directory for the extension. Test the extension with cfx test. A browser window will open and close. No interaction is possible.

Run the application with cfx run. A browser window opens with the extension enabled. Interaction is available. Close the window to end the run.

Create the XPI file for the extension with cfx xpi.

After modifying the extension, are tried to upload it to the builder site. I received the warning that the identifiers did not match.

I modified the id property in the package.json file in the root of the extension. I grabbed the id from the install.rdf from the expanded XPI file for the old extension. The id from the old extension had @jetpack at the end. I removed @jetpack to get the id for the package.json file. I recreated the XPI file and was able to add it as a new version for my extension.

Conduit Search Virus

3/24/14

I have been downloading a lot of files. Somewhere along the way, I picked up the Conduit search virus.

I ran Windows defender: it did not find it.

I ran AVG: it did not find it.

I downloaded AdwCleaner and ran it. It found it and removed it.

11/20/16

Another virus. If I double clicked on a folder, a search started and never finished. I could not open an admin window, I could not run regsvr /i shell32.dll.

Kaspersky did not find it.

AdwCleaner found it and cleaned it.

Insert Key on iMac

I have an iMac with the small, aluminum keyboard. I do not have an insert key. I do not have a help key.

fn+enter simulates the insert key when I use bootcamp and Windows 7.

Many-to-many Database Relation in Yii

I have three database tables:

Phonebook - employee contact info.
Council - members of the Unit that are in the advisory council.
Offices - titles of all offices in the council

Council has fields for an employee id and the year of council service (and a primary key).
Offices are titles (and a primary key).
Phonebook has a field for employee id (and a primary key). It also has fields such as name, address, phone.

An index table, council_offices, has been created to link Council to Offices. A person on the council can hold many offices. Several people can have the same title. It has a field for the Council primary key and a field for the Offices primary key (and a primary key). A relation has been created for each field.

Each table is represented as a model in Yii. The relations method is of importance here. The Offices table does not have any relations.

CouncilOffices relations

Each record in the CouncilOffices table has a many-to-one relationship with the Council and Offices tables. Council and Offices are related as many-to-many. The way to implement this is with two many-to-one relationships through an intermediate table.

BELONGS-TO indicates that CouncilOffices is the many side and Council is the one side of the relation. The council_index field links to the primary field of the Council table. Create a foreign key in the council_offices table that links to the primary key from the council table.

A similar relationship exists for Offices.

        return array(
                    'councilRelation'=>array(self::BELONGS_TO, 'Council', 'council_index'),
                    'officesRelation'=>array(self::BELONGS_TO, 'Offices', 'offices_index'),
        );
   
Council relations

Council defines the many-to-many relationship to Offices. Offices does not define any relations. The third parameter defines how the relationship works: It uses the council_offices table, tying the council_index to the offices_index. These names are the actual names from the table, including the table name and field names. It is confusing that Offices is the name of the Yii class for the model, while council_offices is the actual name of the table in the database.

Council also defines a relationship to the phonebook. It should actually be one-to-one, but one-to-many will also work.

        return array(
                    'officeRelation'=>;array(self::MANY_MANY, 'Offices',
                        'council_offices(council_index,offices_index)',
                        ),
                    'phonebookRelation'=>;array(self::BELONGS_TO, 'Phonebook',
                        'id',
                        ),
        );

Create a foreign key from the id field of the council table to the phonebook table. In my case, the foreign key is to an index, not to a primary key. I am using mySql with INNODB tables. Yii does not specify this relationship, it only exists in the actual table. Create an index for the target field in the phonebook table and then use SQL to create the foreign key (phpMyAdmin only allows foreign keys to primary keys).

alter table council add foreign key (id) references phonebook(pid);

Phonebook relations

The Phonebook does not need any relations in Yii. It should have an index for the empolyee id (pid) in the phonebook table.

Form that accesses data

The payoff is accessing the relations. In the example below, $data is the CouncilOffices table.

Using Ping in Windows

The ping command from a Windows command prompt can be used to check the availability of a host:
Windows documentation.

Two options control how many pings to send. The default number is four.

ping -t

Keep sending Echo Requests until interrupted.

ping -n 5

Send the specified number of Echo Requests.

Two options can be used to test the size of the maximum transmission unit. Use one option to specify the size of the message to send and use the other option to prevent the message from being broken into smaller sizes. If the message is too large to send then the router will not send it.

ping -l 1500

Specify the size of the data field in the message to send.

ping -f

Do not fragment the message. The message must be sent as one unit or be rejected.

ping -f -l 1500

Combine both flags to test the size of the transmission unit. Try smaller numbers for the size until the message is sent.

One option can be used to find the time it takes to reach the host. It can also be used to find intermediate relay addresses.

ping -i 10

Specify the time that the message can live - time to live (TTL). If the time expires, then the message will not be sent. It may be that the message makes several hops before the time out is reached. The last name server to process the message may send a response indicating that the request timed out.

Start with a small number for the time to live and increase it until the host can respond.

Start with a TTL of 1 and you will get a response from a close name server to your machine. Increase the TTL by 1 on subsequent requests and you will get responses from different name servers along the route. Some requests may time out, since the name server that received the timed out packet did not forward the failure notification. Other servers will send a message that the request timed out.

The last procedure can also be used to check the name servers along the route. By increasing the TTL, different name servers will respond with messages. The flaw is that name servers are not required to send a response when a message expires. This is the procedure that the tracert command does.

Another option can be used to find the name servers along the route.

ping -r 9

Record the name servers that forward the message. The count must be in the range from 1 to 9.

Padding a numeric field with zeros in Excel

Use the Text function in Excel to pad a column with zeroes.

To pad to seven places, use the formula =TEXT(A1, "0000000").

Copy the values of this new column back to the original column, in the event that the original column is referenced in other formulas.

Converting Excel Date to mySql Date

I read about this trick from
http://blog.mclaughlinsoftware.com/2009/06/16/excel-date-conversion/

The trick is to convert the Excel date to the mySql format using the Text function.

=Text(A1,"yyyymmdd")

I want to keep the original date in the excel sheet, so I created a new column in the table and filled it with the above formula.

Next, I saved the spread sheet and then saved it again as a CSV file.

I removed the column names from the file, copied the column with the formula back to itself as values, and deleted the original date column.

I now have my original Excel file and a copy as a CSV that has the correct date format.

I then imported the CSV file into the mySql table.