A company finds grievance against some poor soul because he is hosting up everything “Metallica”.  Sounds familiar right, so let’s call the company hired to sue this poor kid, “Payne & Fear”. Okay, now unbenounced to you,  you just happen to be downloading some recently un-copyrighted Elvis Presley, right?  Well one of these Elvis files is also shared from your friend the Metallica pirate (I know strange tastes in music but follow along).  So the company “Payne & Fear” that gets hired to hit our friend and all his followers with a cease and desist (Sue’Em All).  Well you too get hit with a cease and desist because the law office of Payne & Fear simply see you doing your P2P thing with this Metallica fan.   The cease and desist letters go out to all his followers.  Also, Get-This!  These letters are not only cease and desist anymore they are fines that simply state your guilty and you need to pay our company Payne & Fear thousands of dollars or else we will take you to court and sue you for much, much more.  The real scum bags here are these jerky sue happy Lawyers.  Okay, seriously now for a second Google Search “lawyers hunt down p2p” or simply click here.  read a couple of them, not far from the truth am I?  How about another quick search for “RIAA Fines for copyright infringement music” 

Well what is better than P2P, torrents and Limewire?  Old School, Usenet is.  Why because it has a legitimate purpose and if you use Secure Socket Layer (SSL) no one will ever know what your doing.  Yeah, sure your IP address is still connected to a place where piracy occurs however so does a lot of other legitimate downloads.  They might suspect you however someone would likely need a search warrant to prove what you are truly doing with your connection to a Usenet service.

Also for the record UseNext is crap I’m talking about Usenet.  Net I say boy, UseNET, not next!Usenet is one of the oldest computer network communications systems still in widespread use. It was conceived in 1979 and publicly established in 1980 at the University of North Carolina at Chapel Hill and Duke University,^[1] over a decade before the World Wide Web was developed and the general public got access to the Internet.  The Wikipedia page on Usenet  does great justice in documenting the history of this little known service that started over 30 years ago.

Usenet was a free service carried by about 50% of the internet service providers until about 2005 – 2006 when ISP’s started to drop the little known services for several reasons the furthermost reason being the frequent piracy and child pornography that occurs from the service.

Today most of us will require a Usenet provider to access the service of which I’ll only mention one.  There are several providers you need only search for “Usenet provides” to begin finding one.  Please note the Usenet provider service will be in addition to your Internet Service Provider.  You need both providers to have access.  Additionally you will want software that communicates with today’s Network News Transfer Protocol (NNTP).  Again I’ll only mention one such program where there are several.  You need only search for “nntp client software” to discover the different programs that exist.

So here I’ll document setting up connection between the NewsBin Pro software (news client) or aka the NNTP application and the http://www.giganews.com/ Usenet service.   Obviously if your new to both of these you want to start by downloading a 10 day trial copy of the software from http://www.newsbin.com/ and setting up a paid for account on http://www.giganews.com/ .  I’ll also mention here that apparently giganews allows and pays for referrals I just found this out! really after write this blog!  So if you click this link and purchase giganews I get some kind of a bonus credit!  yeah!!!   I guess you also need my real name..  Donald Hughes

http://www.giganews.com/?c=hughesdo

The fine people at Giganews have done a great video on configuring the NewsBin Pro software.  Simply watch the video below and skip ahead to 4:56 to fast forward directly to the NewzBin configuration section.

So everything is installed and configured how do I go about downloading stuff like crazy?  Well you could start download right away by adding a select newsgroup and downloading the headers.  However I’m going to take you to a much simpler route of going to yet one more web page that indexes these headers into easily searchable and single click download(s).  Again there are many of these however I’m only going to mention one.  You could search for other similar services by searching on “Usenet index nzb”.  The key here is the service allows a download of a dot “.nzb” file. This file is a xml file that indexes all of the individual parts that are required for a single item. This is confusing, I’m going to explain this just a little and then have you try it.  The files that are made available on Usenet are composed of much smaller messages.  The original intention of Usenet was for threaded discussions of flat ascii text.  So any one download is likely composed of several discussion threads.  Each required discussion posting needed to complete a single download is a xml record with in one .nzb file download.  In short if you download these .nzb files  NewzBin Pro will in-turn queue-up and download the entire program your requesting.

So, for a couple of bucks you will also want to join http://www.newzbin.com .  This web site allows you to search for individual programs and simply make one click to download an entire program or movie. The web site is quite busy looking at first.  Here below I’ve done a simple search for “Dragon NaturallySpeaking” against Everything where there is a Newzbin type file and selected Go.  In the results I’ll click a green little down pointing arrow ( )  to select the program to download.  I suggest that you read the comments by hovering the “X comments” URL prior to making your selection.  Other users that have downloaded and installed the program will inform you if the program works correctly contains a keygen or if any virus is present.

The site also maintains there own help guide here : http://docs.newzbin.com/index.php/Newzbin:Basics

Well now you are queuing-up downloads like crazy.  Depending on your speed of your Internet you’ll be downloading tons of stuff at the fastest possible speeds.  You will no-longer be depended on other people participating in a P2P torrent to get faster download speeds. 

Finally there are a couple of smaller programs you are going to need but don’t worry they are free to use and easily installed with little to no configurations.  The files you are likely to download from Usenet will perhaps be archived with Winrar, simply click here to download the free version.  Additionally your going to run into places where parts of the files are not 100% completed.  This sounds much worse that it is.  Most people contributing to the Usenet are also creating additional parity files.  Parity files are extra files that you hopefully do not need however these files exist to correct problems with missing portions of files.  So if you do get a file that will not completely download and shows a partial missing red spot within the file’s download monitor go ahead and use Newsbin Pro to write the bad file anyhow and then download the extra parity files. These parity files will fix the original archive.  You’ll need to download and install QuickPar first.  So you may as well do this download now before you need it, click here to download and Install QuickPar.  Once the program is installed you can simply double click on any .par file to begin examining the entire archive’s completeness and begin repairing it if necessary. This can take a little bit of time.  Also note that Newzbin does handle quite a bit of this repairing and un-archiving itself if you have the latest versions and it’s properly configured.  However it is still worth installing the two small programs Winrar and QuickPar.

Let me quickly end this posting in a old school train of thought and a moral one.  If you use a program you should own a legal copy of it.  Being a programmer myself I expect to be paid for my work.  I think the question of fair use comes to mind. Let’s quickly look at that…

Section 107 of the fair use laws contains a list of the various purposes for which the reproduction of a particular work may be considered fair, such as criticism, comment, news reporting, teaching, scholarship, and research. Section 107 also sets out four factors to be considered in determining whether or not a particular use is fair:

1. The purpose and character of the use, including whether such use is of commercial nature or is for nonprofit educational purposes
2. The nature of the copyrighted work
3. The amount and substantiality of the portion used in relation to the copyrighted work as a whole
4. The effect of the use upon the potential market for, or value of, the copyrighted work

The distinction between fair use and infringement may be unclear and not easily defined.

I’m not a lawyer and I hope that I’ll never need one.  However if you already own every Metallica CD ever made and you are simply downloading a MP3 formatted version for your favorite Metallic tune then is this fair use?  I think so.  If you are using a new copy of Microsoft Visual Studio to teach yourself a little about programming and not actually running a business from your creations or use of the program is this fair use?  Let’s be serious here, if you installed any such a program for less than 30 days of research is this fair use?  I myself continually research software to learn and consult with businesses about the best possible solutions for their particular problems.  I then wind-up promoting and evangelizing the merits of these programs to people who wind-up buying them.  I actually need a pay check for myself from a few different software companies for doing such a good sales job on their behalf.  Boy, if I had a nickel! Perhaps I should sue Microsoft for selling their products and not getting paid?  Hmmm….

Still not promoting piracy here just trying to tell you that it is also a two way street.  Another old school though about piracy.  Piracy is a huge promotional tool.  You see if you allow a little piracy in the beginning of your product launch then the product promotes itself.  Your Crazy you say??, okay let me put it in today’s prospective. Companies have been selling you the printer for next to nothing for a couple of years now.  However the Ink costs you an arm and a leg, right?  This is how they make the money for the printer product. By making a “computer chipped cartridge” that knows it is out of ink and needs replaced. Right?  I saw a new printer commercial where the Ink is now cheap.  Why did a company come about to this change?  Well too many of us know about “china ink cartridges”, pirate or knock-off Ink cartridges, whatever you want to call them.  These knock-offs are so cheap we don’t buy from the original manufactures therefore little money is being made on the cartridges.  So while the little known ability to pirate the cartridges in the beginning is a good thing it becomes bad as too many people learned about the availability of these knocked-off cartridges.  What made some of us buy a printer that was the fact of knowing that it was going to be cheap now and in the future as we purchased knock-off Ink cartridges.  But as so many people join our once little known ways the companies had to make printers that cost more and have cheap Ink.

Okay, other examples: 

http://www.mathewingram.com/work/2008/01/24/books-does-piracy-now-marketing/ Obvious This guy know what I’m saying.

http://en.wikipedia.org/wiki/DIVX_(Digital_Video_Express) This would have worked and beaten the DVD format because geeks in-the-know where going to easily hack this format. The only reason it failed was the hack factor was just not great enough. 

http://www.codinghorror.com/blog/2008/05/revisiting-the-black-sunday-hack.html Direct TV’s Black Sunday Kill.  Pirate’s are told “Game Over” stop pirating our sat signals we are too big now and we really just want your money. Plus there are too many of you, however thanks for promoting us and we hope you enjoyed the couple of free fight nights that you invited all your friends to watch.  Now that they too have bought Direct-TV we are going to stop the piracy.

I have recently run into several Cache-Control issues within some web pages here at work.  I wanted to document how the issue has presented itself and how the issues have been resolved so that anyone encountering the issues elsewhere could quickly resolve.

Caching is a well known concept in computer science: when programs continually access the same set of instructions, a massive performance benefit can be realized by storing those instructions in RAM. This prevents the program from having to access the disk thousands or even millions of times during execution by quickly retrieving them from RAM. Caching on the web is similar in that it avoids a roundtrip to the origin web server each time a resource is requested and instead retrieves the file from a local computer’s browser cache or a proxy cache closer to the user.

The Caching issue usually presents itself with a end user stating the page is not taking my updates or that after my update this popup continues to appear even after I have left the page.  These don’t directly indicate that a caching issue exists further investigating would be needed.  If the page in question really did make the update, then you might have a caching issue.  Additionally, some programmers set-up popup dialogs based on events.  Another way the issue has presented itself to me was a html popup window that will not close.  My theory is, that after a popup occurs let’s say, the Modal Pop-up dialog (“check has been successfully deleted”) then JavaScript must have incremented a variable that is being tested to turn off the Pop-up.  The problem that occurs is that page posts back to itself and on the end user’s computer it receives the “cached” version from the hard drive and NOT the web server.  Therefore the JavaScript variable is therefore the previous setting that does not turn off the Pop-up. 

Without doing anything extra to any web page the page is generally allowed to be cached by most browsers.  generally speaking you actually have to do extra work to not allow a page to be cached.  I’ll get to this extra work here in a second but first I want to mention that a server can be set up so that inverse of this is true. On a IIS server you can change the default IIS Settings to add a custom Cache-Control header to a hierarchy of sites or folders.   For example at the root level of your web site you could add the Cache-Control header and therefore all folders above this one and all the web pages will include a “Cache-Control:no-cache” (picture below).

For a very long time our servers had a header placed at the root to not allow caching of any web page with a custom header and we specifically set some web pages to allow caching where it was needed.  We simply removed the headers from certain pages after the more global root setting was made. The reasons some pages can not contain a no-cache setting is that they actually require the ability to be cached.  For example, you can not prevent caching on an “application/octet-stream” download on a SSL encrypted site.  Therefore a couple of pages that produced secure adobe files had to be marked without any Cache-Control headers by visiting this node or file on the hierarchy and specifically deleting the header.  This is easily done however IIS server 5.0 appears to reset to the root settings from time to time.  I think some IIS server updates cause this reset.  At any rate our company has now encrypted files written to the laptop hard drives and I by default allow caching mostly anything from our IIS servers by default.  In real geek speak most every page browsed today is a “response 304” that indicates that the file in the cache is valid.  it really depends on the user’s browser setting if the cached version is used (follow along)..

On different browsers the end user has some control over their computers ability to use a cached version of web pages.  This option exists in Internet Explorer in Tools/ Internet Options / General/ Settings (Internet Explorer 8.0 shown below).  Actually deleting the users cached files could resolve the immediate issue however a far better alternative is to have the server not allow the page to be cached in the first place. 

Where and why would you actually want a page to never be cached?  I’m Glad you finally asked..

A page that’s method is to “get” or “post” back to itself with a saved set of changes in general should be cached-controlled with a no-cache option.  By this I mean this page should never be cached in the first place!  The obvious thing that is noticed when a page does not have this no-cache option is that the page did not take your changes or at least it appears that way to the end user.  On the post-back the cached page is display by the browser instead of waiting for the round trip to complete.  This might also be especially true of compressed pages.   So the end user winds up seeing the page that was originally display to them upon the first visit without the updated fields.  You ask why would the programmer have not seen the same thing in testing.  Testing might have been done on a separate box with different settings and additionally the programmer might have different browser settings for the temporary storage of files (cache) or finally the server admin simply turned on IIS compression recently and the page now caches differently.

How might you actually make the changes to web page that have a Cache-Control issue? 

If the page in question has a .aspx page with a header I would make the three meta tag addition in red below.

<%@ Page Language="VB" AutoEventWireup="false" CodeFile="Default.aspx.vb"

    Inherits="_Default" %>

<%@ Register Assembly="HtmlEditor" Namespace="HtmlEditor"  TagPrefix="cc1" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"

    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >

<head runat="server">

    <title>HR Self-Serv Contenet Manager</title>

    <META Http-Equiv="Cache-Control" Content="no-cache">

    <META Http-Equiv="Pragma" Content="no-cache">

    <META Http-Equiv="Expires" Content="0">

    <link href="StyleSheet.css" rel="stylesheet" type="text/css" />

</head>

....

If it could not be done in the .aspx page but could be done in the .vb page or otherwise need done in compiled dll I would simply include the dot net directive SetCacheability as shown in code below:

If myReader.Read Then

    Response.Clear()

    Response.BufferOutput = False

    Response.ContentType = "text/html"

    Response.Cache.SetCacheability(HttpCacheability.NoCache)

    Response.Write("<link href='StyleSheet.css' rel='stylesheet ….

You could go nuts with all the different headers that potentially modify the page’s cacheability such as these:

Response.ClearHeaders();

Response.AppendHeader("Cache-Control", "no-cache"); //HTTP 1.1

Response.AppendHeader("Cache-Control", "private"); // HTTP 1.1

Response.AppendHeader("Cache-Control", "no-store"); // HTTP 1.1

Response.AppendHeader("Cache-Control", "must-revalidate"); // HTTP 1.1

Response.AppendHeader("Cache-Control", "max-stale=0"); // HTTP 1.1

Response.AppendHeader("Cache-Control", "post-check=0"); // HTTP 1.1

Response.AppendHeader("Cache-Control", "pre-check=0"); // HTTP 1.1

Response.AppendHeader("Pragma", "no-cache"); // HTTP 1.1

Response.AppendHeader("Keep-Alive", "timeout=3, max=993"); // HTTP 1.1

Response.AppendHeader("Expires", "Mon, 26 Jul 1997 05:00:00 GMT"); // HTTP 1.1

However I would stick to the two examples above.  Also the program fiddler could be used to verify that a 304 Not-Modified response header is not being returned, 304 indicates that the file in the cache is valid and can be used again as opposed to getting a 200 response.