Find URL Description

We can find a lot of code in C# that will parse the http urls in given string. But it is difficult to find a code that will: Accept a url as argument, parse the site content Fetch all urls in the site content, parse the site content of each urls Repeat the above process until all urls are fetched.