Cloaking/Hiding/Filtering Unwanted Directories During Checkout (aka "Partial Checkouts") With Subversion 1.4.x

The Problem:

Ever find yourself working with a large-ish SVN repository that includes many sub-projects, and you only want to work on a subset of those sub-projects, but you don't want to deal with all the extra cruft involved in checking out at the top level of the repository?

"What's this guy talking about?"

Sorry, a picture will help me explain.  Below is a snapshot of an old repository of one of products we develop here at work:

As you can see, trunk consists of multiple sub-components, each of which have their own branches/tags/trunk folders.  Whether or not this is the best repository layout could be up for debate, but it seems to treat us well with the design of this project. (We have a repository layout similar to this for another project we have, and it seems to serve us well there as well.)

The big problem we have with this repository layout is that you never want to checkout that top-level trunk.  You end up with all of the sub-branches and tags folders that are not only superfluous, but also waste space and make the commit or update process so much slower because SVN must check all the sub-folders for modifications.

What ends up happening, for example, is someone needs to work on the "Bids" and "Main" components at the same time.  To facilitate this without checking out at the trunk-level is they create two separate working folders, and check out each respective component's trunk (or appropriate branch).  When they commit, they have to make two separate commits.

While this works, it's not desirable, especially if you aspire to reach Continuous Integration nirvana. Not being able to create an atomic commit across various working folders could potentially break things in a CI environment if changes in one component rely upon the changes of another.  Even if CI isn't your eventual goal, an atomic commit of 'grouped changes' just makes sense.

"What to do?" 

The Fix:

Well, the good news is that Subversion 1.5 is supposed to have better support for 'partial checkouts'.  I'll admit I haven't kept up with that aspect of v1.5, to be totally honest, I'm more excited about merge tracking. But that's for another post.

But anyway, there is a work-around in SVN 1.4.x (and probably earlier versions..sorry for not doing any fact-checking here.)  Well, maybe work-around isn't the right word, it's more of an off-label use of a repository feature.

Before continuing on, you must know that I'm primarily a Windows developer.  I use Subversion on Windows via the excellent TortoiseSVN front end. The following how-to is going to be using TortoiseSVN.  Everything that follows is entirely possible with the command line SVN tools, and, if there's enough request, I'll update this post with how to do it from the command line. (email me!)

"Get to it already, would ya?"

Sorry, here we go, in short:

1. Create a repository on your local machine.
2. Checkout a working copy of the repository and specify an svn:external property that will link to the portions of the remote repository you really want to work with. (Read the SVN Book for important notes and details.)
3. Commit this svn:externals property back to the local repository.
4. Update you working copy of the local repository and you'll have checked out the parts of the remote repository you want to work with.

Details:

Create a new folder somewhere on your machine, make sure it's in a place that you won't forget, and in non-obtrusive location because once this is created, it can't be moved without causing you pain in the process. ;)

MyFilterRepo is a folder on my desktop.  Now, using TortoiseSVN, create an empty repository inside that new folder.  Right click on the folder, choose  TortoiseSVN -> Create repository here...  Choose the 'FSFS' file system type.

 

Then, browse this repository, right-click the folder, TortoiseSVN->Repo-browser:

Create a 'trunk' folder (or any other folder...it doesn't really matter:)

Create another empty folder some place.  Checkout the trunk folder from the repository you just created by specifying the file:///bla/bla/bla/trunk style URL:

Now, specify the svn:external properties that will link your local repository's trunk folder to multiple remote repository folders. To do this, right-click on your working folder, and choose TortoiseSVN->Properties. In the dialog that pops up, click the 'Add' button. This will bring you to this dialog:

Choose 'svn:externals' From the Property name combo box. In the Property value text box, you will need to enter key/value pairs of the remote repositories.  The first item on the line is what you want folder to be named in your local repository.  The second item on the line is the full path to the remote repository.  (If you have credentials already cached for the remote repository, you shouldn't have a problem, otherwise you'll probably be asked to specify your username/password when you update later.)

When you hit the "OK" button, you'll notice that your working folder has been changed:

 

Now, commit your working folder back to your local repository.  Don't worry, the repository(ies) linked via the svn:externals property will not be modified at this time. 

 

Now, update your working folder.  You'll notice that all of the repository locations mentioned in your svn:externals property will be checked out to your working folder.  Hooray!  You did it!

You are now able to work in the sections of the remote repository that you want to work on, while completely ignoring all the other cruft that you don't want to see or care about. When you commit any changes to this local repository, you'll be pushing those changes to the remote repository in a single atomic commit. 

(I must admit, I originally found this via tortoisesvn.net, but since it's been one of those issues that I've dealt with for so long, I decided I had to duplicated the work-around here. )

To further understand how this works, I must again point you to the section of the official SVN book that describes externals, and what they're really intended to be used for.

Let me know if this helps!

 

Update - Gotchas

Surprisingly, there are very few gotchas with this technique. Thankfully, the only ones I've discovered are easily worked-around.
So, you have multiple externals defined, and you've made changes to the various external 'parts', like so:

Normally, to commit those changes, you'd go to the parent folder, and choose to commit from there. The problem is, as you'll notice, is that TortoiseSVN will give you a message like:

You'll also notice that the list of modified files doesn't include your changes. Notice how the message states that you'll have to commit those changes separately. If we'd RTFM'd like we were supposed to, we'd notice the paragraph near the bottom of Chapter 7, Section 3 of the SVN Book states: "So, for example, if you want to commit changes that you've made in one or more of those external working copies, you must run svn commit explicitly on those working copies—committing on the primary working copy will not recurse into any external ones."
This is exactly what we're trying to not do! Thankfully, there's a work-around which allows us to still create atomic commits across those externals:

1. Go into the the working folder
2. Select each folder that has modifications that you want to commit.
3. Right-click -> SVN Commit
4. TADA

Now, what we haven't ran into, but I imagine could theoretically happen is, what if one of those externally-included sources itself has external-definitions? For example, in the sample repository we've been using here, what if the DBManager had an external definition? If you made modification inside that nested external definition, there would be no way for you to commit the changes inside that nested external definition, and the changes in your home-spun externals repository.

Dell / EMC AX150i/AX150SCi and Windows Server 2003 x64 iSCSI Initiator Woes

Can I have a much larger post name? 

In summary, this post details troubles I experienced by doing the right thing: I RTFM'd, and the manual was wrong.

I'm not sure how many people this post will help, but I know I'm writing it under the same guidelines I usually follow for posting: I had a hard time solving a problem, and the internet at large really didn't offer me much help, so, here we are.

At work we recently purchased a new server and storage-area-network (SAN) device:

• Server: Dell 2950
  • Dual Intel Xeon 5320's (quad-core!) (1.86Ghz, 4MB cache, 1066MHz FSB) - 8 cores!
  • 16GB RAM
  • 6 160GB hard drives - RAID5'd via the integrate PERC 5/i RAID card. (I have some gripes about the PERC 5/i that I may describe in a later post.)
  • Dual integrated gigabit ethernet adapters.
• SAN Device: Dell / EMC AX150i SP (ie: EMC AX150SCi)
  • iSCSI interface
  • Single storage processor (ie: "SP" in Dell-speak, "SCi" in EMC-speak)
  • 8 250GB HD's in a RAID-5 config (with space for 4-more drives, and the ability to swap-in 750GB drives!)
  • Separate 8-port Dell PowerConnect 2708 gigabit ethernet switch for the SAN network backbone / 'fabric'.

The 2950 is great, we're got it running Windows Server 2003 R2 Standard x64 Edition (this is important for later...) We have another 2950 in the office, and the only complaint I have with both of them is the server management software's inability to notify you, in any useful way, of RAID-failure events.  This is actually a problem with the integrated PERC 5/i card, but that's not what this post is about read Eric Neale's post about the PERC 5/i for more details on that gripe.

The SAN is pretty great too, except it's setup software is giving me problems, but before we get to that, a little more information on the configuration, and steps I took to get to the problem.

I'm going to be using the AX150 in a less-than-super redundant configuration.  I have the PowerConnect 2708 switch configured to be the primary switch for a private network that is used only for this storage-area-network.  The 2950 does not have an iSCSI host bus adapter (HBA), it's going to connect to the SAN with it's secondary NIC. Installing the necessary software on the Windows side of this was very easy.  Getting that software configured to work with the AX150 wasn't so.

I've followed EMC's instructions to-a-'T', which pretty much proceed as:

• Install the Microsoft iSCSI Initiator software
• Install the latest EMC PowerPath software
• Install the latest EMC Navisphere Server Utility
• Install the latest EMC Navisphere Initialization Utility
• ...then follow the configuration steps.

I downloaded and installed v2.0.5 of the MS Software Initiator - followed EMC's specification that I install the 'Initiator Server' and the 'Software Initiator' - but not the 'MPIO Multipathing  Support for iSCSI.' This gave me an icon in my control panel and on my desktop.  EMC's documentation didn't state anything about configuring it at this point, so I left it alone. (yeah, that's foreshadowing.)

I then downloaded and installed EMC's PowerPath v5.1.0 from their registration-only support site.  This required a reboot to finalize installation.

Next up I downloaded and installed  v6.20.4.2.0 of the Navisphere AX Server Utility. This is where I ran into a couple separate issues.  In the box of the AX150 there was a big-poster-style "Getting Started" guide that came with the AX150.  At the point where it says to install the Server Utility it states to install with the defaults, and to make sure that the 'Registration Server' is installed - which it states is the default.  But, it was disabled by default when I ran the installer.  The documentation I retrieved directly from EMC's web site stated that I should explicitly not install the 'Registration Service'.  Okaaaaaay. So, which is correct? First time through, I choose not to install it because I figured the online documentation was more recent/up-to-date. The second snag in this installation started when it asked me:

"Are you installing this utility on a server that is using the Microsoft iSCSI initiator to connect to the CLARiiON storage system?"

Neither the 'big-poster' or the online documentation stated what I should choose, so, since it sounded appropriate, I choose 'Yes'. It then presented this message/error:

"Microsoft Initiator is needed to setup your iSCSI devices. InstallShield does not find Microsoft Initiator installed on your computer. Please download Microsoft Initiator from www.microsoft.com and install it."

Whaaaaaaaat?  I had already installed MS iSCSI Initiator!  Strangely though, the installation didn't abort, and continued on to completion with no other errors or strange messages.

After this, I download and installed v6.20.0.3.11 of the Navisphere Initialization Tool. After this is installed, I was instructed to use it to initialize the AX150. The initialization process assigns IP addresses to the management interface and iSCSI ports on the unit. 

After initialization, I was instructed to open the Server Utility and choose the "Configure iSCSI Connections on this server" option. Well, look at the screenshot below and you tell me what I should choose:

 

Yup.  There's my problem summed up in a screenshot.  Figuring I had missed something, or done something wrong, I meticulously combed through my installation notes, and cross-referenced the installation documentation. I tried rolling back installations and re-installing with different options. (Specifically, I tried installing the 'Registration Service', and I also tried answering 'no' to the Server Utilities install question mentioned earlier.)  No combination of installation options made a difference. I spelunked through more documentation on EMC's support site.  I looked on Dell's site for documentation -- they kept pointing me to EMC's support site.  I tried Google for answers - I found nothing. 

I finally decided I'd have to get in touch with Dell.  I went to their support page, got to their 'online chat support' area, entered the AX150's service-tag, and, tada, found that everyone was busy with someone else.  (It's nice to see chat-support is no-more available than phone support.)  I tried a couple times, hoping to get through.  Nothing.  I really didn't want to email them because I needed to get this system online, and who knows how many days turn-around it would take to get resolved via email. 

I started looking for the correct phone number to call, and I somehow, probably because I'd entered the AX150's service tag, I found myself at a tech-support page specific to the AX150.  There's a section near the top: "Top Solutions : Frequently Asked Questions About Your Model":

1. What is RAID 0 and RAID 1
2. What is RAID 5
3. How can I collect the SP event logs from a CLARiiON FC4700 or CS-Series array using the serial port?
4. How do I configure the LUNs on my DELL|EMC® array?
5. EMC® Navisphere® Server Utility iSCSI configuration is not available in Microsoft® Windows® 2003 64-bit

(Side question: What's with questions #1 and #2?  "You must be at least this smart to ride this ride." comes to mind. hehe.)

Number five!  Look at that!  That's exactly my problem! Dell's documentation had the answer, not EMC's!  Dell keeps redirecting you to EMC's documentation when they had the answer to my question all along. 

It's a bummer that I had to spend this much time finding this answer because getting things configured manually through the MS iSCSI Initiator is really quite simple.

Oh well...live-n-learn.

Getting Breakpoints to work in Eclipse PDT using Zend Debugger

November 18th, 2007 UPDATE: Please see the section at the bottom of this post for an update!

I've recently been fooling around with PHP.

At work we have a Drupal-based web site, and I figured it was time to get a better mental-grasp around PHP. After looking through a whitepaper from IBM titled Using open source software to design, develop, and deploy a collaborative Web site I decided I needed to get myself a development environment setup in which I could work with Drupal 'under-the-hood'.

IBM's whitepaper was written in various pieces over the course of about a years time. One of the sections discusses setting up Eclipse with different plugins that facilitate PHP development. In a later section they mention the PHP Development Tools (PDT) - a more comprehensive set of PHP development and debugging tools for Eclipse. Lucky for me, they now have a PDT all-in-one Eclipse package that makes setup even easier. (Get the package from here.) After downloading and installing it, and getting a feel for the whole thing (wow, Eclipse is complicated at first,) I got down to the brass-tacks and tried debugging.

Before I go on, here's a little more information about my development environment:

• OS: Window XP SP2
  
• Development Environment:
  
  • Eclipse : PDT
    
  • PHP 5.2.4
    
  • MySQL 5.Something
    
• Web server: Apache
  

PHP, MySQL and Apache are all from the Apache Friend's XAMPP package.

I followed the instructions on the PDT Wiki for instructions on how to Install the Zend Debugger for Eclipse.



Just for reference, in the screenshot above, you'll notice my "Dummy Debugger Test Server" configuration. Here's a screenshot of the specific details of that config:


After getting everything setup, I could get the debugger to debug any PHP-based web page -- so long as I had the "Break at First Line" debugging option set. I could start the debugger and at the first line of PHP code, the debugger would 'catch', and allow me to step through my code. Everything worked as expected - variables and their values were visible in the Variables dialog, I could step into and out of various functions. Unfortunately, if I set breakpoints anywhere, they would never 'hit', the debugger wouldn't halt the PHP execution on the specified breakpoints. It would zip right past them as though they never existed.

Naturally I hit-up my good friend Google for answers. I found a number of people having the same problem as myself. I found instances of people having this problem all the way back in January of 2007. I also found instances of people having this problem as recently as October of 2007! Every single time I found someone saying they had the problem, nobody ever had an answer for them!

Finally, finally, I was able to find this somewhat cryptic post on how someone fixed the problem:

Not sure if this helps, but I ran into the same problem myself. I found out that it was that the plugin expects the project root to be the http root.

My project structure had the http root as a subfolder of the project root.

If the http root isn't the same point as the project root, then the plugin can't correctly inform the debugger which file the breakpoint is in.

I ended up hacking the source to make the plugin work at a subfolder level.

Problem was, he stated that he hacked the source of the plugin to 'make the fix'. Not exactly what I was hoping to do. I figured I was approaching yet another dead end, but something about him stating that the plugin was simply confused about expecting your Eclipse project's root to be the web server's root gave me an idea: Move my content to the virtual host's root directory.

I tried moving my project's files all into the root folder of the web server (just dummy.php in this case.) I modified my "Dummy Debugger Test Server" configuration as shown in this screenshot:


I made sure that "Break at First Line" was unchecked/disabled, and fire up the debugger. It worked. It finally worked. After all that trouble it freakin' worked.




Has anyone else had this much trouble? Has anyone been able to get debugging to work when your debugged PHP doesn't reside in the root of your web server?

November 18th, 2007 Update:
I think I may have a better example. I've also been 'playing' around with the Symfony PHP Framework. I again ran into this breakpoints-not-working problem when trying to debug a Symfony-based project.

I'm going through the "Askeet" tutorial and have imported a project into Eclipse and had my Apache instance serving up the "/web" subfolder served up as the server's document root.

Of course, this confuses the debugger plugin, because, again, the debugger is assuming that the Eclipse project-root corresponds directly to the web server's root directory, which it's not. This is the situation:



The arrow titled "Assumed" is what the debugger is wrongly assuming. To fix the problem, you have to configure your web server to match that assumption.

So, I modified the virtual host's DocumentRoot to point to the Askeet project's root folder, even though there is no index.html or index.php to be served from there. I had to also modify the "/sf" alias to handle "/web/sf". (If you're using Symfony, you should know what I'm talking about.) So, now instead of going to http://localhost, I have to go to http://localhost/web/. But since this is strictly for development purposes, it's not a big deal. Oh, and I had to make a slight modification to the debugging profile in Eclipse to match the new /web/index.php URL. Again, not a big deal.

I hope this update cleared any confusion...

TortoiseSVN's "Noisy" TSVNCache.exe

A co-worker of mine pointed me to a post by Travis Illig on his blog titled "Optimize Tortoise SVN Cache (TSVNCache.exe) Disk I/O"

A short synopsis:

TortoiseSVN has a shell-overlay that indicates the status of your files and folders within an SVN 'working folder'.

The background process that 'watches' your file system for SVN-related files and folders is TSVNCache.exe.

TSVNCache.exe, according to Travis, was bogging his system down with disk I/O.

Instead of disabling TSVNCache.exe entirely (and thereby disabling the icon-overlays) he found a way to limit what TSVNCache.exe 'watches' on the file system.

When I first read his post, I thought "How bad can TSVNCache.exe be?" So, as noted in the comments of Travis' post, I grabbed FileMon from SysInternals, and fired it up. After setting a filter so it would only display TSVNCache.exe-related events, I was amazed at just how much 'noise' it was causing -- and I wasn't doing anything at all! Just sitting there watching FileMon do it's thing.

What really caught my eye was when I sent my coworker an instant message, thanking him for the link. My IM client wrote the message to it's log file, and I saw TSVNCache note the change, and check the log file's location for the existence of a ".svn" folder. (The existence of such a folder would indicate that the folder is an SVN working folder.)

I was surprised to say the least, and wanted to see how bad it really got. I 'putzed ' around doing a whole lot of nothing other than navigating around my file system. I seemed ridiculous at just how much disk I/O TSVNCache.exe was inducing.

For example, look at the following screen shot. It's the TSVNCache activity imposed when I copied a folder on my desktop, which contains one file:



Thankfully, by limiting which folders TSVNCache.exe 'watches' you can greatly limit this superfluous disk I/O. In case you still haven't taken a moment to click-through and read Travis' post, here's the quick run-down:

1. Open your TortoiseSVN settings (Right-click the desktop, TortoiseSVN -> Settings)


2. Go to the "Icon Overlays" item in the treeview.


3. Add "C:\*" to the "Exclude Paths" box


4. Add the folders you do want included in the "Include Paths" box
   
   • I entered my 'work' folder: "C:\Work\*", for example.


5. Save changes : Apply/Ok


6. Restart TSVNCache.exe - kill TSVNCache.exe in your Task Manager. It will automatically restart when you open a Windows Explorer.

With these changes in place, within FileMon, you'll notice that TSVNCache is a lot less noisy. It will still receive "Change Notify" messages, but you won't see it access files/folders outside of your specified "Include" list.


Thanks for the great tip, Travis!

Diff / Merge Tools

I've been using TortoiseMerge (part of TortoiseSVN) for a while now to view my source's diff's as well as handling merge conflicts. Something about it though has always felt 'clunky' to me. It may just be it's default color scheme that causes my mental detraction, but when I saw Larkware plug something call DiffMerge 3.1 beta I figured I should give it a try.



So far, I like it. It's UI is more sexy, as far as I'm concerned. It supports 3-way merging (like TortoiseMerge), making life so much easier when resolving merge conflicts. It supposedly supports folder diff/merging, although I haven't tried that yet.

I just wish I hadn't also accidentally stumbled upon another free diff/merge tool called KDiff3. I like KDiff3's shell context menu for it's ability to 'remember' a file's location on the file system. Once a file is 'remembered', you can perform a diff on any other file on the system by asking KDiff3 to compare the current file you have in front of you, and one of the 'remembered' files from earlier. (I've found myself needing to do this lately and KDiff3 definitely speeds up the process.)

One other thing I liked about KDiff3 was in it's installer: It provided an install option to integrate with TortoiseSVN as Tortoise's primary diff/merge tool.

The only problem with KDiff3 was, it's color scheme made me throw up in my mouth so fast that I quickly decided I had to give DiffMerge another shot. (The screenshot shown here doesn't truly show off the vomit-in-mouth colors so much...) KDiff's ability to integrate easily with TortoiseSVN got me looking into how to set DiffMerge as my Tortoise diff/merge tool.

Being perpetually low on free time, and not wanting to figure out the command line options myself, I enlisted Google to give me answers. In my search for integrating DiffMerge as my TortoiseSVN diff/merge tool, I ran into this post by 'Trumpi' detailing exactly how to integrate the two! So, instead of outright stealing from Trumpi, I give you: Trumpi's blog : SourceGear release DiffMerge.

...
Ok, I'll steal a little, only because I tweaked my settings slightly for the diff-tool specification (caught the tweak in Trumpi's post comments). Here's the command lines for specifying DiffMerge as your diff and merge tools for TortoiseSVN:

• Diff:
  
  • New way: (see comments)
    C:\Program Files\SourceGear\DiffMerge\DiffMerge.exe /ro2 /t1=%bname /t2=%yname %base %mine
  • Old way:
    C:\Program Files\SourceGear\DiffMerge\DiffMerge.exe /t1=Base /t2=Mine %base %mine


• Merge:
  
  • New Way: (see comments)
    C:\Program Files\SourceGear\DiffMerge\DiffMerge.exe /r=%merged /t1=%yname /t2=%bname /t3=%tname /c=%mname %mine %base %theirs
  • Old Way:
    C:\Program Files\SourceGear\DiffMerge\DiffMerge.exe /t1=Mine /t2=Base /t3=Theirs /r=%merged %mine %base %theirs

Update:

Note on the updated TortoiseSVN external tool commands: In the comments, Travis mentioned that I should RTFM. Upon inspecting the DiffMerge manual, I came up with the updated command lines above. I especially appreciate the "/ro2" option for the diff-tool. This makes DiffMerge act strictly like a diff-tool (ie: no editing allowed.)

Also, I gave Jimmy's WinMerge a try. It does fit my 'eye-candy' requirement, but I just can't devote more time to it because of it's lack of 3-way merging.

Google Earth update

Hey, did anyone else notice the cool additions they made in Google Earth?

Check Out The Details Here.

Of special note, you can now flip-around and view the sky from your current location on Google Earth! This is really cool. There is also a built-in flight simulator. It's pretty easy to crash in the flight sim, so, I didn't really play much with it, just the 'view sky' mode.

Anyway... Quick post!

Cool SIGGRAPH 'Content Aware' Image Resizing Demonstration

This is pretty stinkin' cool:

I Have Adam's Book!

Just wanted to say that I got Adam Nathan's COM Interp book, and I'm already taking notes as to what I need to blog about. :) It's already answering questions that I've had, and while I thought what I was recently reading was going to give insight into the problems I had previously, it just skirted the issue and said "More in Chapter 7, 20, and 24".

This book is huge. We're talking bigger-than-the-Bible huge. So big, they wouldn't bind it with just one binding. No, they bound it as two seperate tomes. Tomes of arcane COM knowledge. ;)

Although I'm only into chapter 4, I can tell this book has what I need. Heh, well, I've also heard that if this book doesn't have the answers to your COM-interop questions, then no other book will, soooo. :) It hasn't been updated for .NET 2.0/3.0/3.5. I'm curious to know if any of the version-specific remarks Adam makes regarding the implementation of the SDK at the time of his writing have changed. I would guess the answer is "not much" because who would miss the chance to write a second edition!? :D

Anyway, before I go, does anyone else think it's kinda weird the way I just happened to have "N" post-it notes for Adam *N*athan's book?  

VMWare : Loosing eth0 after you've copied your VM

Background

Here's a bit of weird-behavior I've noticed when working with some of our production virtual machines (running Gentoo Linux) here at work.

In order to update the OS's on our virtual machines, I will copy them to my local machine, power them on, update them, and then push the updated OS's back out into production at the earliest convenience.

When you copy the VM from one location to another, VMWare notices this and asks you "Hey, it looks like this machine has been physically moved or copied, do you want me to create a new VM-UUID?" If you answer in the affirmative, VMWare internally regenerates any unique-identifiers tied to this virtual machine. The one thing that's really noticeable is that any virtual ethernet adapters get their MAC addresses changed.

The problem I've experienced is that when you power on the new-UUID'd VM, you no longer have an ethernet adapter. Gentoo tries to bring-up eth0 and it says "network interface eth0 does not exist" and "Please verify hardware or kernel module (driver)"

Explanation

"So, what's going on?"

Try a couple things:

• If you run lspci you should still see the ethernet adapter.


• If you run 'dmesg' and should see the kernel find the network card and it even calls it eth0

"So, where does eth0 go?"

Try running ifconfig -a. I bet you now have an eth1 and it's MAC address matches the newly-generated virtual MAC address specified in the virtual machine's .vmx file.

"Oh great, so every time I copy the VM I need to update the system configs to use the new eth1, or eth2, etc!?!?!"

No, hush, I'm getting to the answer.

The problem stems from the linux distro 'remembering' the MAC address of the network adapter and expecting it to be the same between boots. In the case of our Gentoo VM's, it's udev that mucks this up.

"Ok, fine, it's udev's fault. We know it's broken because it's expecting the ethernet adapter to have a MAC address that it no longer has. What to do?

The Answer

To fix this problem you need to tell your linux distro the VM's new MAC address. How you do this can vary by distro. In my spelunking, I found a few ways:

• In Gentoo do one of the following: (Do #1, it's the easiest.)
  
  1. Delete /etc/udev/rules.d/70-persistent-net.rules and reboot. Your eth0 should be back.
     • 2007/09/13 Update: This almost-always works for me. But, for some reason, sometimes it seems to confuse udev even more; after rebooting, I'll have an eth2 or eth3. When this happens, I end up following #2, making sure the udev config file has 'eth0' listed, and not eth1, eth2, or eth3.
  2. Edit /etc/udev/rules.d/70-persistent-net.rules (or whatever it's named) to match your new MAC address and reboot. Your eth0 should be back.
• Other distros:
  • Look for, (and edit if you find,) /etc/iftab
  • Look for, and delete, then reboot /etc/udev/rules.d/25-iftab.rules
  • Look for, (and edit if you find,) /etc/sysconfig/network-scripts/ifcfg-eth0

Give Credit Where Credit Is Due

I got hints from a number of pages, but in the end, it was the folks over at the VMWare discussion forums for the win:
VMWare Discussion Forums

Marshalling Arrays To VB6's COM Funland pt2

Ok, yesterday I figured out how to get an array of interfaces (actually, objects that implement an interface, but go along with my sloppy grammer, ok?) out of C#, through the COM Callable Wrapper (CCW) and into COM-land.

Wouldn't you know it, but I also need to be able to pass an array of objects back into C# from COM-land.

Well, I'm here to say that figuring this out was a lot easier than yesterday's problem. At first I tried something like:

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
[ComVisible(true)]
public interface ITest {
   void TakeBusinessObjects(IBusinessObject[] bos);
}

Well, that doesn't work. In VB6, when you try to compile something that calls TakeBusinessObjects(...) you get the a compile error like the following:

Function or interface marked as restricted, or the function uses an Automation type not supported in Visual Basic

Well, my Google digging on that particular error actually proved fruitful and the answer is simple:

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
[ComVisible(true)]
public interface ITest {
   void TakeBusinessObjects(ref IBusinessObject[] bos);
}

(credit goes to Jon Wojtowicz for his Using COM Callable Wrappers to Extend Existing Visual Basic 6.0 Applications post at EggHeadCafe.)

Hooray! That works! The array gets passed back into .NET-land, and things seem happy. Except, I wouldn't be writing this post if I didn't have a problem, right? Well, TakeBusinessObjects(...) doesn't throw an exception, and it reaches it's return statement successfully. Unfortunately, something gets lost in translation while returning control to VB6-land because I intermediately upon returning, VB6 raises this error:

Class does not support Automation or does not support expected interface
Number: 430

This error makes it sound like I've developed on v2 of some COM component, but I've deployed the compiled EXE on a machine that only has v1 of the COM component. What I don't understand is that the array gets passed through the CCW into .NET-land! It works! Something just goes wrong on the way back to VB-land.

Update: The Answer! (Kind of)

I gotta hand it to my buddy Jimmy - that guy has given me the "Try XYZ" that has fixed whatever problem I was tackling so many times -- and he's come through once again! He suggested that, I take a look at the [In] and [Out] attributes.

By default, when I compile my COM component, it's being assumed that I not only want to be able to take in an array reference, but that I also want to push any changes made to that array reference back out to the caller. Well, lucky for me, I don't make any changes to the array once it's in .NET-land, and I can flag the parameter as only needing to come into the method, and not out. Like this:

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
[ComVisible(true)]
public interface ITest {
   void TakeBusinessObjects([In] ref IBusinessObject[] bos);
}

So, while this fixes most of the situations where I would need to pass an array from COM-land into .NET-land, it still irks me that I don't know why VB6 throws that error if the CCW marshaller tries to move the array back out of .NET-land when the method returns.

Marshalling Arrays To VB6's COM Funland

Ok, so, please, don't ask why I've found myself writing a COM component in C#, and am using it in VB6. Just accept that fact that I need to.

This COM component needs to return an array of data to VB6. At first I thought "Why not just have my C# project reference the VB6 runtime via interop, and I'll be able to instantiate a VB6 Collection class, and return that to VB6, and it will be happy.

But no, when I try to instantiate a VBA.CollectionClass I get:

Retrieving the COM class factory for component with CLSID {A4C4671C-499F-101B-BB78-00AA00383CBB} failed due to the following error: 80040154.

After much googling, the closest answer I could get was that I was trying to instantiate a VBA.Collection on an x64 platform. Um, the last time I checked my Pentium-D was a 32 bit processor. I even went so far as to force my C# COM component to compile to x86 specifically instead of the 'Any CPU' configuration. Still, the error persisted.

So, I had to bail on that idea, and come up with "Hey, what about just returning an array? What a great idea! I performed a test.

I created a test COM visible interface and class, something like:

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
[ComVisible(true)]
public interface ITest {
   String[] GetStrings();
}

[ClassInterface(ClassInterfaceType.None)]
[ComVisible(true)]
public class Test : ITest {
   public String[] GetStrings() {
      List foo = new List();
      foo.Add("hello");
      foo.Add("bye-bye");

      return foo.ToArray();
   }
}

Then, the corresponding test-code in VB6:

Dim testObject as Test
Dim strings() as String

set testObject = new Test
strings = testObject.GetStrings()

It worked.

I was happy, and I continued working on my C# code as planned.


Days later, I have hundreds of lines of code down in C#, and I'm at a good point to test what I've written. Now, my objects weren't going to be returning string-arrays like the test case above, rather, they're going to be returning arrays of an interface defined in my C# COM component, something like:

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
[ComVisible(true)]
public interface IBusinessObject {
   Int32 GetSomething();
}

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
[ComVisible(true)]
public interface IFoo {
   IBusinessObject[] GetBusinessObjects();
}



[ClassInterface(ClassInterfaceType.None)]
[ComVisible(true)]
public class BusinessObject : IBusinessObject {
   /* you get the idea */
}

[ClassInterface(ClassInterfaceType.None)]
[ComVisible(true)]
public class Foo : IFoo {
   public IBusinessObject[] GetBusinessObjects() {
      List bos = new List();
      /* Add some BusinessObjects to 'bos' */

      return bos.ToArray();
   }
}

Then, the corresponding VB6 code:

Dim foo as Foo
Dim myObjects() as BusinessObject

set foo = new Foo
myObjects = foo.GetBusinessObjects()

Unfortunately, it didn't work. While no exception was thrown in C#, somewhere between return bos.ToArray() and VB6's assignment to the myObjects array a "Type Mismatch" error was thrown in VB6. I couldn't figure out why, though.

I tried catching the array returned from .GetBusinessObjects() into a Variant like:

Dim myObjects as Variant
myObjects = foo.GetBusinessObjects()

I still received the "Type Mismatch" error. I was really lost here, because in VB6-land a Variant is the closest thing you're going to get to a generic object pointer as you're going to get.

I again didn't garner much assistance from another thorough Google spelunking. In my searches, I did stumble across the MarshalAs attribute, but since I'm not a COM-master I wasn't entirely sure what I should be marshaling an array of interfaces as in order to safely reach COM-world. I blindly tried a number of things, and always ended up with "Type Mismatch". I was loosing hope. (As a side note I desperately need to get a copy of Adam Nathan's book .NET and COM: The Complete Interoperability Guide.)

Finally, I stumbled upon it, and I'm not entirely sure why I didn't try it first:

The Answer!

[InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
[ComVisible(true)]
public interface IFoo {
   [MarshalAs(UnmanagedType.AsAny)]
   IBusinessObject[] GetBusinessObjects();
}

What's weird is that this produces a compiler warning making it sound like the attribute is not of any use:

Type library exporter warning processing 'MyNamespace.IFoo.GetBusinessObjects(#0), MyProject'. Warning: Type contains [MarshalAs(AsAny)], which is only valid for PInvoke. The MarshalAs directive was ignored.

If you place the attribute on the actual implementation you do not get the compiler warning...but you also get the Type Mismatch again. So, it would seem the warning is ignorable as it is applying to something.

Something tells me that once I get my hands on Adam Nathan's book, this will become a lot more obvious to me, and/or I'll find a better answer. Either way, I've got a solution for now.


So, why this post? Mostly as a note-to-self as to how to solve the problem in the future, but there's the hope that some poor sap such as myself has had the exact same problem and that this will turn up for them while they spelunk Google.

If this proves useful to anyone, please, drop me a line.

Weird SQL2005 TempDB Table and Primary Key Behavior

Ok, here's something I ran into the other day while working with some temp tables. I needed a temp table. That table was probably going to be bloated, and could benefit from a primary key and some extra indexes on it since I'd be doing some heavy queries against the table. The SQL used in this post isn't the same SQL as from my app, rather, it's been simplified to the point of demonstrating my problem.

Consider the create-table statement below:

Figure 1

CREATE TABLE #myTmp (
   [id] [int] NOT NULL,
   [id2] [int] NOT NULL,
   [foo] [decimal](6,3) NOT NULL,
   CONSTRAINT [PK_myTmp] PRIMARY KEY CLUSTERED ([id] ASC, [id2] ASC))

Ok, so this gives us a very simple temp table that's scoped to the connection (or stored procedure) that it's create inside. It also has a primary key.

I'd been designing this temp table, and testing my SQL in a query window inside Management Studio. I had run this SQL, and the query window and it's connection were still open when I copy-n-pasted the code into my application and took it for a test run.

I got an error:
"There is already an object named 'PK_TmpWorking' in the database."

Whaaaaaat? It's as though the name of the primary key doesn't follow the same scoping rules as other objects created in TempDB! This seems really weird to me, and my google-spelunking didn't turn up many answers (am I loosing my touch?) The only way around this was to not name the primary key constraint, like so:

Figure 2

CREATE TABLE #myTmp (
   [id] [int] NOT NULL,
   [id2] [int] NOT NULL,
   [foo] [decimal](6,3) NOT NULL,
   PRIMARY KEY CLUSTERED ([id] ASC, [id2] ASC))

This leaves SQL Server to come up with a name for the primary key, and it appears that it names it some random jibber-jabber.

So, while I can't really logic-out an answer as to why I'm seeing this behavior, I do at least have a work-around. But get this, as if this primary-key thing isn't weird enough, it would seem this 'bug' only exists for primary keys, and not other named indexes. What I haven't shown you is that, in my query window, I also had a number of CREATE INDEX statements, similar to:

Figure 3

CREATE NONCLUSTERED INDEX [IX_MyTmp_Foo] ON #myTmp ([foo] ASC)

I had this executed in my query window. The temp table had it's randomly-named primary key, and it had this named index. When I ran the same code in my application, there was no complaints at all. I would have expected to run into the same "There is already an object named 'IX_MyTmp_Foo' in the database." error, but I didn't get one!


Does any of this make sense? Can anyone explain why it appears that primary keys on temp tables aren't scoped within the same scope as the temp table itself? Especially since regular indexes appear to be scoped as you would assume.

Signed:

Confused in Yooperland.

Oh wowwwww...

Ok, in this day in age, it's difficult to make me so awe-struck that my jaw hits the floor. Well, I finally got around to checking out the Tech Preview of Microsoft Live Labs' Photosynth, and let me tell you, my jaw is still on the floor. Totally awesome technology.

I'm going to give you guys two links, one that will take you to a MS Live Labs blog post containing a video of a guy at TED (Technology, Entertainment, Design) Conference in Monterey, California explaining a couple of new MS-acquired technologies. Photosynth is included. Find that post here.

Then, go and install and try Photosynth yourself. You will need to install the browser plugin to use it.

Eye Strain

Ok, nothing like posting only to link someone else, but here goes!

I found 22 Ways to Reduce Eye Strain at Your Computer via LifeHacker.

Now, I've stared at computer screens for a fairly large portion of my life, and only in the last couple years have I started developing eye-strain related ailments (head aches, and occassional blurred vision.)

Other things I found via the above links is How To Exercise Your Eyes, and a program called WorkRave. WorkRave may just replace my home-spun 'egg timer' application that bugs me at defined intervals to "'stand up, relax your eyes, etc...".

Enjoy!