IIS worker processes
For a project I wanted to make optimal use of the significant server resources I had: 16 cpus and 24Gb of RAM. I implemented a custom HttpHandler that spawns a lot of requests, and used Apache Ant to request images rendered in 25 parallel threads.
This took quite some time, and I noticed that the server was not actually doing anything; mostly using 1-2 cpu's a little bit during this process. Why wasn't the server capable of parallellization of these 25 requests more efficiently?
Then I found a bit of a hidden setting: Maximum Worker Processes was set to 1 in the used Application Pool. I set it to 12, and now my cpu utilization chart looks like this:

That is a really lovely sight, isn't it? I can sit back and just watch those cpu's burn their cycles...
How to tweet beyond i-am-eating-a-banana
Everybody is twittering nowadays, and too many still use it only to broadcast messages about what they are doing; the "i am eating a banana" tweets. No problem with that, and in fact, I like to know when my friends eat bananas, especially when I'm bored waiting in line or in a train. So, please my friends, keep posting all the strange and stupid things you are doing, and let me be the judge of whether the information should be filtered out or not. I can always unfollow you if your tweets are too much spam.
However, there is so much more value and fun to Twitter that many of my friends do not get. I've written about the incredible value of the "@" sign tweets before here on this blog, and have seen similar great responses from guru's and corporate entities that I would not get using any other medium like email, phone or shouting in the street.
Let me try to explain some of the key elements that I find so valuable on twitter, as sort of a tutorial.
Value 1: how to use the @ sign
The word that follows any @sign is supposed to be a twitter username, and all twitter clients create link to the profile of that user from it. You can have multiple @ signs in a tweet, in any order, anywhere in your tweet. You can add user accounts for people you follow, but also for people you do not follow! So you can comment on famous people to your followers, and that does not mean you want a response from that user. But you could, because any user will be able to see mentions of his @sign on his pages. Famous people will get that a lot, but normal people like us will usually all of them.
If you see a tweet from someone and you want to respons, simply reply by adding their @sign in it, do not use RT for that
Value 2: how to use RT (retweet)
RT or retweet is a practice that is now standard: use it to copy a tweet to your followers if you find a tweet from someone interesting or funny or stupid, just add "RT @user TheMessage" with the whole original message. If you want you can add your comments like "funny! RT @user joke", add your comment in front, otherwise your followers cannot distinguish between your comment and the original message
Value 3: how to use hash tags (the # sign)
Hashtags is also simply a convention. By using twitter search, you can search for tweets with a certain word in it, and even see live streams of these tweets. If you add a hashtag like #os2010 (for the olympics in Dutch) to your tweet, then twitter clients will create a link to a search for other tweets with this word in it. This way people who are interested in that subject can easily find your tweet as well, and your message (or question!) will be seen by many more than just your followers (and often get responded to, or retweeted)
You can invent your own tags, but many events, conferences, and even tv shows will have their own hash tag; look them up and use them.
Value 4: how to use lists
Information overload is a problem of course, but the list feature of twitter is very handy there: you can group people you follow in lists that you give a name, like "friends" and "fun" and "news" and "work"... you can then use a twitter client like TweetDeck or just the links on the web for a list to pay more attention to some lists than others, so that one valuable tweet from your mother is not drowned in a flood of banana spam.
Note that you can also follow a list of someone else! And that you can see in what lists you are from others. Have you used lists already as a good way to find new interesting people to follow?
Value 5: how to use twitter to read news on a subject
One of the great things about twitter is keeping up-to-date on a subject, you can follow news twitter feeds of course, but its better to follow interesting people, those people that tweet about new memes, funny video's or news about your industry first.
I think it is imperative that everyone has a daily search feed open in TweetDeck or something like that with the topic that interests them that day, be it the tool you're using, the music artist, the movie you're going to, or whatever. Every day, and you can change it any time of course. Keep it in a separate column.
Value 6: how to use twitter to get help or advice
If you need advice or help on a subject, just ask the question precisely, and use the correct hashtags. You'll be surprised how often you get a valuable response, including a new person to follow. Some hashtags are interesting specifically, like #durftevragen or area codes like #020 in The Netherlands
Value 7: how to use twitter to get real-life feedback during events
This is where the fun starts. Never feel alone anymore. When you're on a conference, use the hashtag to tell what you think, and ensure you follow a live search feed on it to see what the others are thinking right now. If you're watching a tv show, find out the hash tag people are using, and follow that. Big events like the Olympics, or the council elections #GR2010 we just had in The Netherlands are so much more fun to watch while having a live twitter search feed on your screen.
Well there's much more to Twitter, hopefully this tutorial has helped you a bit. I'm sure that when you start using these tips, you will find out that the social network of Twitter is much more powerful than you would expect, and will help you in your work and day-to-day life in a way that would not be possible by email, the web or IRL
5 lessons I learned today on image processing in .net
I'm working on a web application to create a large set of tiled images based on an immense data collection. For that really great project (can't show you what it is though until it's live) I had to refresh my image processing skills again, after I've done several image sites before.
I chose to use ASP.NET as the application, with a virtual immense canvas, working with C# 3.5, and still staying with the standard GDI+ classes that are included. I learned a few things by trial and error that I want to write down because I will forget them and need them a year from now. And maybe it's helpful for you too.
Things I learned about image scaling in .net:
Lesson 1: Loading an image without holding on to the file handle is hard
Simply loading an image from disk is not as straightforward as it may seem. Yes, there is
Image img = Image.FromFile("myfile.jpg");
But that solution has some drawbacks: first, it keeps a lock on the file long after you've disposed of the image, so you cannot delete the images on disk while the application is still alive, even if it's only reading it. A workaround solution I found on the net works like this:
FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
Image img = Image.FromStream(fs);
fs.Close();
But that has another issue: it can crash your application completely while doing processing with the image with the superb error message:
A generic error occurred in GDI+
Fine... Well, I learned that this is indeed related to the first: the image classes in .net use a form of lazy streams, so the constructed Image object from the stream actually needs access to the stream later on, depending on what you do with it (and what type of image you have). So it is not safe to close the stream until you are really ready with the Image too. But that was hard for me because of how I set up my classes... So if you do not close the stream you're OK, but you still cannot remove the file underneath, and you have a memory leak...
The answer is to load the file into a MemoryStream first, and then use that stream to create the image; then you can actually close the first file stream, and leave the memory stream open (this will be problematic if you do not have enough memory of course)
Lesson 2: how to speed up controlled image loading from disk
Furthermore, there's a key set of parameters to the Image.FromStream, where the second boolean parameter is called validateImageData (default set to true), and it indicates whether the system should check if the image file is valid. If you know your image sources, you can set it to false, and the loading from disk (which was my main performance bottleneck) will speed up dramatically!
Unless of course, you have a specific type of TIFF image, which means that the validation and conformation of the image, if not done at load time, will be done when you draw the image later... so not that much help there, but it did help a lot for the image tiles that I created later on
Lesson 3: even though the image extension is .jpg, the mimetype is still called image/jpeg (with the extra 'e'
Lesson 4: how to best control jpeg quality
Jpeg images are great because you can control their quality and thus their compression and disk or web size. This is controlled by a number from 1 (poor) to 100 (highest quality). Note this does not influence the quality of the actual photo or art work you're working with, just the quality of the image.
I used a quality of 50 (as a standard compromise) like this:
public static ImageCodecInfo GetCodec(String mime)
{
foreach (ImageCodecInfo codec in ImageCodecInfo.GetImageEncoders())
{
if (codec.MimeType == mime)
{
return codec;
}
}
return null;
}
protected static EncoderParameters GetEncoderParameters()
{
EncoderParameters paramz = new EncoderParameters(1);
paramz.Param[0] = new EncoderParameter(Encoder.Quality, 50L); ;
return paramz;
}
public static void SaveImage(System.IO.Stream s, Image img)
{
using (Profiling.Go("SaveImage to stream"))
{
img.Save(s, GetCodec("image/jpeg"), GetEncoderParameters());
}
}
Which works really well. By the way, did you notice the immensely great profiling code that automatically works with hiearchical thread detection? That's something I created and it helps me every day... it's better than anything you've seen. Contact me for a demo
Lesson 5: Aspect ratio's are hard
Yes, there are more aspects to aspect ratio than you know of. I now know all there is to know about aspect ratio. ;-)
Well, that's it for today. A great day of learning with progress. The tool will be very great and sexy and no doubt get a lot of attention when it's out there, I will surely inform you when it's live and send you the link.
