Jump to content
PeterPanettone

Reliable method for getting the FavIcon URL?

By PeterPanettone, in ICS - Internet Component Suite

Recommended Posts

PeterPanettone    182

PeterPanettone
Posted

I am using Google’s favicon service (https://www.google.com/s2/favicons?domain=...) to get the FavIcon URL of a website/internet-domain (e.g., https://www.cnpack.org -> https://www.cnpack.org/favicon.ico), as the FavIcon URL is not always Website URL + /favicon.ico.

 

Unfortunately, Google seems to apply rate limits to this: When doing this in a batch for many different websites/internet-domains, there is a rate limit warning after 55 or 56 requests:

 

image.png.44fcfb4d3a8a2321d34306b2df2fcf92.png

 

My question is: Does the ICS library have a reliable method to retrieve the FavIcon URL of a domain that is not subject to rate limits?

Share this post

Link to post

DelphiUdIT    271

DelphiUdIT
Posted (edited)

You can scan the head section of html for a line with "icon" like:

  

<head>
	<meta http-equiv="content-type" content="text/html; charset=windows-1252"/>
	<title>TITLE OF THE SITE</title>
	<link rel="icon" href="https://www.mydomain.zz/favicon.ico" />             <--------------- THIS
	<meta name="created" content="00:00:00"/>
	<meta name="changed" content="2022-02-20T12:09:39.579000000"/>
</head>

 

Edited by DelphiUdIT

Share this post

Link to post

PeterPanettone    182

PeterPanettone
Posted
6 minutes ago, DelphiUdIT said:

You can scan the head section of html for a line with "icon"

Thanks for the friendly hint. However, IMO, this is a guessing and is far from reliable when scanning many different sites.

 

Currently, I am using this function, which leads to the warning when requested several times: 

function GetFavIconURL(const AURL: string): string;
const
  DEFAULT_SIZE = 64; // Define a default size
var
  DomainOnly: string;
  //UrlParts: TURI; // TURI is a record, not a class, so no Free needed
begin
  DomainOnly := System.NetEncoding.TNetEncoding.URL.Encode(AURL);

  // It's good practice to URL-encode the domain in case it has unusual characters,
  // though for standard domains it's often not strictly necessary.
  // For modern Delphi (10.3+), you can use TNetEncoding.URL.Encode (add NetEncoding to uses).
  // For older Delphi versions, you'd use TIdURI.URLEncode from Indy.
  // DomainOnly := TNetEncoding.URL.Encode(DomainOnly); // Uncomment if needed

  // Now, construct the Google Favicon service URL using the extracted domain:
  Result := Format('https://www.google.com/s2/favicons?sz=%d&domain=%s', [DEFAULT_SIZE, DomainOnly]);
  //Clipboard.AsText := Result;
end;

 

Share this post

Link to post

Angus Robertson    696

Angus Robertson
Posted

The short answer is ICS has no special way of accessing Google APIs to overcome rate limits. 

 

I thought that most web sites simply placed the favicon.ico file in the root directory so easy to find, not sure of the benefit of adding a link in web pages, unless you want a different icon for different pages.  

 

But using ICS to access a specific web page to get that link from the header is trivial. 

 

Angus

 

Share this post

Link to post

PeterPanettone    182

PeterPanettone
Posted
3 minutes ago, Angus Robertson said:

The short answer is ICS has no special way of accessing Google APIs to overcome rate limits.

Hi Angus, thanks for the answer. I didn't ask for a method to access Google APIs, as I already have one. Does the ICS have ANOTHER technique to get the FavIcon URL?

Share this post

Link to post

PeterPanettone    182

PeterPanettone
Posted (edited)
On 10/16/2025 at 2:33 PM, DelphiUdIT said:

You can scan the head section of html for a line with "icon" like:

  


<head>
	<meta http-equiv="content-type" content="text/html; charset=windows-1252"/>
	<title>TITLE OF THE SITE</title>
	<link rel="icon" href="https://www.mydomain.zz/favicon.ico" />             <--------------- THIS
	<meta name="created" content="00:00:00"/>
	<meta name="changed" content="2022-02-20T12:09:39.579000000"/>
</head>

 

After guessing the FavIcon URL (https://www.cnpack.org -> https://www.cnpack.org/favicon.ico), we can check if it exists: 

function FileExistsOnWeb(const URL: string): Boolean;
var
  HTTPClient: System.Net.HttpClientComponent.TNetHTTPClient;
  Response: System.Net.HttpClient.IHTTPResponse;
begin
  // Default to False - assume file doesn't exist until proven otherwise:
  Result := False;

  // Create HTTP client component (nil = no owner, we'll manage memory manually):
  HTTPClient := System.Net.HttpClientComponent.TNetHTTPClient.Create(nil);
  try
    try
      // Increase max redirects to handle multiple server redirects
      // Default is 5, which may not be enough for some sites:
      //HTTPClient.MaxRedirects := 10;
      HTTPClient.MaxRedirects := 20;  // Very generous limit

      // Enable automatic handling of redirects (should be default, but explicit is better):
      HTTPClient.HandleRedirects := True;

      // Set a reasonable timeout to prevent hanging (in milliseconds):
      HTTPClient.ConnectionTimeout := 5000;  // 5 seconds
      HTTPClient.ResponseTimeout := 5000;     // 5 seconds

      // HEAD method requests only HTTP headers, not the file content
      // This is much faster than GET because it doesn't download the actual file
      // For a favicon.ico, we save downloading ~1-50KB of data:
      Response := HTTPClient.Head(URL);

      // Status code 200 means "OK" - the file exists and is accessible
      // Other common codes: 404 (Not Found), 403 (Forbidden), 500 (Server Error):
      Result := (Response.StatusCode = 200);
    except
      // Catch network/HTTP exceptions (timeout, DNS failure, connection refused, etc.):
      on E: System.Net.HttpClient.ENetHTTPClientException do
        Result := False;  // Any exception means we can't confirm file exists
      // Catch any other unexpected exceptions:
      on E: Exception do
        Result := False;
    end;
  finally
    // Always free the HTTP client to prevent memory leaks
    // This runs even if an exception occurs:
    HTTPClient.Free;
  end;
end;

 

Edited by PeterPanettone

Share this post

Link to post

Lars Fosdal    1961

Lars Fosdal
Posted

It's not quite trivial to "guess" the icon...

I wonder if the Chromium source contains any clues to how it is done in Chrome?


DelphiPraxis header contains:  

<link rel='shortcut icon' href='https://en.delphipraxis.net/uploads/monthly_2018_11/favicon.ico.020df3ba12dd9fb453b85b3c3cee72ab.ico'>

Mastodon header contains: 

<link href='/packs/assets/favicon-16x16-74JBPGmr.png' rel='icon' sizes='16x16' type='image/png'>
<link href='/packs/assets/favicon-32x32-CiQz7Niw.png' rel='icon' sizes='32x32' type='image/png'>
<link href='/packs/assets/favicon-48x48-DMnduFKh.png' rel='icon' sizes='48x48' type='image/png'>
<link href='/packs/assets/apple-touch-icon-57x57-BsPGHSez.png' rel='apple-touch-icon' sizes='57x57'>
<link href='/packs/assets/apple-touch-icon-60x60-CQE7yLDO.png' rel='apple-touch-icon' sizes='60x60'>
<link href='/packs/assets/android-chrome-72x72-9LRpA3QN.png' rel='apple-touch-icon' sizes='72x72'>
<link href='/packs/assets/apple-touch-icon-76x76-BPRp9FS0.png' rel='apple-touch-icon' sizes='76x76'>
<link href='/packs/assets/apple-touch-icon-114x114-Ch7jwTNh.png' rel='apple-touch-icon' sizes='114x114'>
<link href='/packs/assets/apple-touch-icon-120x120-W9xwzzUZ.png' rel='apple-touch-icon' sizes='120x120'>
<link href='/packs/assets/android-chrome-144x144-D-ewI-KZ.png' rel='apple-touch-icon' sizes='144x144'>
<link href='/packs/assets/apple-touch-icon-152x152-s3oy-zRw.png' rel='apple-touch-icon' sizes='152x152'>
<link href='/packs/assets/apple-touch-icon-167x167-DdVi4pJj.png' rel='apple-touch-icon' sizes='167x167'>
<link href='/packs/assets/apple-touch-icon-180x180-DSCV_HvQ.png' rel='apple-touch-icon' sizes='180x180'>
<link href='/packs/assets/apple-touch-icon-1024x1024-B3Tu3EqI.png' rel='apple-touch-icon' sizes='1024x1024'>

GMail 

<link rel="shortcut icon" href="https://ssl.gstatic.com/ui/v1/icons/mail/rfr/gmail.ico" type="image/x-icon">

Undisclosed site 

    <link rel="apple-touch-icon-precomposed" media="screen and (resolution: 163dpi)" href="/Content/img/iOS-57px.png" />
    <link rel="apple-touch-icon-precomposed" media="screen and (resolution: 132dpi)" href="/Content/img/iOS-72px.png" />
    <link rel="apple-touch-icon-precomposed" media="screen and (resolution: 326dpi)" href="/Content/img/iOS-114px.png" />
    <link rel="shortcut icon" href="/Content/img/favicon.ico" type="image/x-icon" />

Zaptec 

<link rel="mask-icon" href="/favicon/safari-pinned-tab.svg" color="#111111"/><link rel="icon" type="image/svg+xml" href="/favicon/favicon.svg"/><link rel="icon" href="/favicon/favicon-32x32.png" type="image/png" sizes="32x32"/><link rel="icon" href="/favicon/favicon-16x16.png" type="image/png" sizes="16x16"/>

 

  • Thanks 1

Share this post

Link to post

PeterPanettone    182

PeterPanettone
Posted

This gave me the highest success rate so far: 

function FileExistsOnWeb(const URL: string): Boolean;
var
  HTTPClient: System.Net.HttpClientComponent.TNetHTTPClient;
  Response: System.Net.HttpClient.IHTTPResponse;
begin
  Result := False;
  HTTPClient := System.Net.HttpClientComponent.TNetHTTPClient.Create(nil);
  try
    try
      // Reasonable redirect limit to prevent infinite loops
      HTTPClient.MaxRedirects := 10;  // Reduced from 20
      HTTPClient.HandleRedirects := True;

      // Shorter timeouts to fail fast on problem URLs
      HTTPClient.ConnectionTimeout := 3000;  // 3 seconds
      HTTPClient.ResponseTimeout := 3000;     // 3 seconds

      Response := HTTPClient.Head(URL);
      Result := (Response.StatusCode = 200);
    except
      // Catch ALL exceptions including redirect errors
      on E: Exception do
        Result := False;  // Silently fail - let caller try next URL
    end;
  finally
    HTTPClient.Free;
  end;
end;

function GetFavIconURL(const AURL: string): string;
// gets the FavIcon URL from AURL
var
  URI: System.Net.URLClient.TURI;
  Host: string;
  PossibleURLs: TArray<string>;
  URL: string;
begin
  Result := ''; // Default to empty string if none found

  URI := System.Net.URLClient.TURI.Create(AURL);
  Host := URI.Host.ToLower;

  // Try multiple common favicon locations in order of likelihood
  PossibleURLs := [
    // 1. Standard favicon (most common)
    Format('%s://%s/favicon.ico', [URI.Scheme, Host]),

    // 2. PNG version (increasingly common, better quality)
    Format('%s://%s/favicon.png', [URI.Scheme, Host]),

    // 3. SVG version (modern, scalable)
    Format('%s://%s/favicon.svg', [URI.Scheme, Host]),

    // 4. Apple touch icon (often high quality, 180x180 or larger)
    Format('%s://%s/apple-touch-icon.png', [URI.Scheme, Host]),

    // 5. Alternative Apple icon name
    Format('%s://%s/apple-touch-icon-precomposed.png', [URI.Scheme, Host]),

    // 6. Google's favicon service as last resort (always works!)
    // Google caches favicons from websites
    Format('https://www.google.com/s2/favicons?domain=%s&sz=128', [Host])
  ];

  // Test each URL and return the first one that exists
  for URL in PossibleURLs do
  begin
    if FileExistsOnWeb(URL) then
      Exit(URL);
  end;

  // If nothing found, return Google's service as ultimate fallback
  // This ensures you ALWAYS get something, even if it's a generic icon
  Result := Format('https://www.google.com/s2/favicons?domain=%s&sz=128', [Host]);
end;

 

Share this post

Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Go To Topic Listing ICS - Internet Component Suite
×