Class UrlUtils


  • public class UrlUtils
    extends java.lang.Object
    • Constructor Summary

      Constructors 
      Constructor Description
      UrlUtils()  
    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static java.lang.String addSource​(Download download, java.lang.String magnet, java.net.InetSocketAddress address)  
      static java.net.Socket connectSocketAndWrite​(boolean is_ssl, java.lang.String target_host, int target_port, byte[] bytes, int connect_timeout, int read_timeout)  
      static java.net.Socket connectSocketAndWrite​(boolean is_ssl, java.lang.String target_host, int target_port, byte[] bytes, int connect_timeout, int read_timeout, boolean unconnected_socket_hack)  
      static void connectWithTimeout​(java.net.URLConnection connection, long connect_timeout)  
      static void connectWithTimeouts​(java.net.URLConnection connection, long connect_timeout, long read_timeout)  
      static boolean containsPasskey​(java.net.URL url)  
      static java.lang.String convertIPV6Host​(java.lang.String host)  
      static java.lang.String decode​(java.lang.String s)  
      static java.util.Map<java.lang.String,​java.lang.String> decodeArgs​(java.lang.String args)  
      static java.lang.String decodeIfNeeded​(java.lang.String s)
      Unfortunately we have code that mindlessly decoded URLs (FileDownloadWindow) which borked (in the case I discovered) magnet uris with encoded parameters (e.g.
      private static byte[] decodeMultiHash​(java.lang.String str)  
      static byte[] decodeSHA1Hash​(java.lang.String str)  
      private static byte[] decodeSHA256Hash​(java.lang.String str)  
      static byte[] decodeTruncatedHash​(java.lang.String hash_str)
      Decodes a hex or base32 encoded sha1, sha256 or multihash
      static byte[] decodeTruncatedHashFromMagnetURI​(java.lang.String hash_str)
      decodes a sha1 or sha256 multihash as found in a magnet URI and truncates to 20 bytes
      private static byte[] decodeTruncatedMultiHash​(java.lang.String hash_str)  
      static void DHHackIt​(javax.net.ssl.HttpsURLConnection ssl_con)  
      static javax.net.ssl.SSLSocketFactory DHHackIt​(javax.net.ssl.SSLSocketFactory factory)  
      static java.lang.String encode​(java.lang.String s)
      Like URLEncoder.encode, except translates spaces into %20 instead of +
      private static java.lang.String encodeName​(java.lang.String name)  
      private static java.lang.String encodeNetworks​(java.lang.String[] networks)  
      static java.lang.String escapeXML​(java.lang.String s)  
      static java.lang.String expandIPV6Host​(java.lang.String host)  
      static java.util.Set<java.lang.String> extractNetworks​(java.lang.String[] magnet_uri)  
      static java.lang.String extractURLHost​(java.lang.String str)  
      static java.net.URL getBaseURL​(java.net.URL u)  
      static java.util.Map getBrowserHeaders​(java.lang.String referer)  
      private static java.lang.String getBrowserHeadersToUse​(java.lang.String encoded_headers)  
      static java.lang.String getCanonicalString​(java.lang.String str)  
      static java.lang.String getCanonicalString​(java.net.URL url)  
      static long getContentLength​(java.net.URLConnection con)  
      static java.lang.String getFriendlyName​(java.net.URL url, java.lang.String url_str)  
      static java.net.URL getIPV4Fallback​(java.net.URL url)
      Returns an explicit IPv4 url if the supplied one has both IPv6 and IPv4 addresses
      static java.lang.String getMagnetURI​(byte[] hash)  
      static java.lang.String getMagnetURI​(byte[] hash, byte[] hash_v2, java.lang.String name, java.lang.String[] networks)  
      static java.lang.String getMagnetURI​(byte[] hash, java.lang.String name, java.lang.String[] networks)  
      static java.lang.String getMagnetURI​(DownloadManager dm)  
      static java.lang.String getMagnetURI​(DownloadManager dm, int max_name_len)  
      static java.lang.String getMagnetURI​(TOTorrent to_torrent)  
      static java.lang.String getMagnetURI​(Download download)  
      static java.lang.String getMagnetURI​(Download download, int max_name_len)  
      static java.lang.String getMagnetURI​(Torrent torrent)  
      static java.lang.String getMagnetURI​(java.lang.String name, Torrent torrent)  
      static java.lang.String getMagnetURI​(java.lang.String name, Torrent torrent, java.lang.String[] networks)  
      static java.net.URL getRawURL​(java.lang.String url)  
      static byte[] getTruncatedHashFromMagnetURI​(java.lang.String magnetURI)
      decodes a sha1 or sha256 multihash from an actual magnet URI and truncates to 20 bytes
      static java.lang.String getURL​(java.lang.String text)  
      static java.lang.String getURLForm​(java.lang.String address, int port)  
      static java.lang.String getURLForm​(java.net.InetAddress address, int port)  
      static java.lang.String getURLForm​(java.net.InetSocketAddress address)  
      static void HTTPSURLConnectionSNIHack​(java.lang.String host_name, javax.net.ssl.HttpsURLConnection con)  
      static boolean isInternalProtocol​(java.lang.String url)  
      static boolean isURL​(java.lang.String sURL)
      test string for possibility that it's an URL.
      static boolean isURL​(java.lang.String sURL, boolean bGuess)  
      static java.lang.String normaliseMagnetURI​(java.lang.String base_hash)
      returns magnet uri if input is base 32 or base 16 encoded sha1 or sha256 hash, null otherwise
      private static java.lang.String parseHTMLforURL​(java.lang.String text)  
      static java.lang.String parseTextForMagnets​(java.lang.String text)  
      static java.lang.String parseTextForURL​(java.lang.String text, boolean accept_magnets)  
      static java.lang.String parseTextForURL​(java.lang.String text, boolean accept_magnets, boolean guess)  
      static boolean queryHasParameter​(java.lang.String query_string, java.lang.String param_name, boolean case_sensitive)  
      static java.lang.String resolveIPv4Host​(java.lang.String url)  
      static java.lang.String resolveIPv6Host​(java.lang.String url)  
      static void setBrowserHeaders​(ResourceDownloader rd, java.lang.String referer)  
      static void setBrowserHeaders​(ResourceDownloader rd, java.lang.String encoded_headers, java.lang.String referer)  
      static void setBrowserHeaders​(ResourceUploader ru, java.lang.String encoded_headers, java.lang.String referer)  
      static void setBrowserHeaders​(java.net.URLConnection connection, java.lang.String referer)  
      static void setBrowserHeaders​(java.net.URLConnection connection, java.lang.String encoded_headers, java.lang.String referer)  
      static java.net.URL setHost​(java.net.URL u, java.lang.String host)  
      static java.net.URL setPort​(java.net.URL u, int port)  
      static java.net.URL setProtocol​(java.net.URL u, java.lang.String protocol)  
      static boolean SSLSocketSNIHack​(java.lang.String host_name, javax.net.ssl.SSLSocket socket)  
      static java.lang.String truncateForURI​(java.lang.String str, int max_utf8_bytes)  
      static java.lang.String unescapeXML​(java.lang.String s)  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • patMagnetSHA1HashFinder

        private static java.util.regex.Pattern patMagnetSHA1HashFinder
      • patMagnetMultiHashFinder

        private static java.util.regex.Pattern patMagnetMultiHashFinder
      • prefixes

        private static final java.lang.String[] prefixes
      • XMLescapes

        private static final java.lang.Object[] XMLescapes
      • last_headers

        private static java.lang.String last_headers
    • Constructor Detail

      • UrlUtils

        public UrlUtils()
    • Method Detail

      • decodeArgs

        public static java.util.Map<java.lang.String,​java.lang.String> decodeArgs​(java.lang.String args)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(byte[] hash)
      • getURLForm

        public static java.lang.String getURLForm​(java.net.InetSocketAddress address)
      • getURLForm

        public static java.lang.String getURLForm​(java.net.InetAddress address,
                                                  int port)
      • getURLForm

        public static java.lang.String getURLForm​(java.lang.String address,
                                                  int port)
      • extractURLHost

        public static java.lang.String extractURLHost​(java.lang.String str)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(byte[] hash,
                                                    java.lang.String name,
                                                    java.lang.String[] networks)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(byte[] hash,
                                                    byte[] hash_v2,
                                                    java.lang.String name,
                                                    java.lang.String[] networks)
      • encodeName

        private static java.lang.String encodeName​(java.lang.String name)
      • encodeNetworks

        private static java.lang.String encodeNetworks​(java.lang.String[] networks)
      • extractNetworks

        public static java.util.Set<java.lang.String> extractNetworks​(java.lang.String[] magnet_uri)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(Download download)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(Download download,
                                                    int max_name_len)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(DownloadManager dm)
      • truncateForURI

        public static java.lang.String truncateForURI​(java.lang.String str,
                                                      int max_utf8_bytes)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(DownloadManager dm,
                                                    int max_name_len)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(TOTorrent to_torrent)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(Torrent torrent)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(java.lang.String name,
                                                    Torrent torrent)
      • getMagnetURI

        public static java.lang.String getMagnetURI​(java.lang.String name,
                                                    Torrent torrent,
                                                    java.lang.String[] networks)
      • addSource

        public static java.lang.String addSource​(Download download,
                                                 java.lang.String magnet,
                                                 java.net.InetSocketAddress address)
      • normaliseMagnetURI

        public static java.lang.String normaliseMagnetURI​(java.lang.String base_hash)
        returns magnet uri if input is base 32 or base 16 encoded sha1 or sha256 hash, null otherwise
        Parameters:
        base_hash -
        Returns:
      • getTruncatedHashFromMagnetURI

        public static byte[] getTruncatedHashFromMagnetURI​(java.lang.String magnetURI)
        decodes a sha1 or sha256 multihash from an actual magnet URI and truncates to 20 bytes
        Parameters:
        hash_str -
        Returns:
      • decodeTruncatedHashFromMagnetURI

        public static byte[] decodeTruncatedHashFromMagnetURI​(java.lang.String hash_str)
        decodes a sha1 or sha256 multihash as found in a magnet URI and truncates to 20 bytes
        Parameters:
        hash_str -
        Returns:
      • decodeTruncatedMultiHash

        private static byte[] decodeTruncatedMultiHash​(java.lang.String hash_str)
      • decodeTruncatedHash

        public static byte[] decodeTruncatedHash​(java.lang.String hash_str)
        Decodes a hex or base32 encoded sha1, sha256 or multihash
        Parameters:
        hash_str -
        Returns:
      • decodeSHA1Hash

        public static byte[] decodeSHA1Hash​(java.lang.String str)
      • decodeSHA256Hash

        private static byte[] decodeSHA256Hash​(java.lang.String str)
      • decodeMultiHash

        private static byte[] decodeMultiHash​(java.lang.String str)
      • getRawURL

        public static java.net.URL getRawURL​(java.lang.String url)
      • getFriendlyName

        public static java.lang.String getFriendlyName​(java.net.URL url,
                                                       java.lang.String url_str)
      • isURL

        public static boolean isURL​(java.lang.String sURL)
        test string for possibility that it's an URL. Considers 40 byte hex strings as URLs
        Parameters:
        sURL -
        Returns:
      • isURL

        public static boolean isURL​(java.lang.String sURL,
                                    boolean bGuess)
      • parseTextForURL

        public static java.lang.String parseTextForURL​(java.lang.String text,
                                                       boolean accept_magnets)
      • getURL

        public static java.lang.String getURL​(java.lang.String text)
      • isInternalProtocol

        public static boolean isInternalProtocol​(java.lang.String url)
      • parseTextForURL

        public static java.lang.String parseTextForURL​(java.lang.String text,
                                                       boolean accept_magnets,
                                                       boolean guess)
      • parseTextForMagnets

        public static java.lang.String parseTextForMagnets​(java.lang.String text)
      • parseHTMLforURL

        private static java.lang.String parseHTMLforURL​(java.lang.String text)
      • encode

        public static java.lang.String encode​(java.lang.String s)
        Like URLEncoder.encode, except translates spaces into %20 instead of +
        Parameters:
        s -
        Returns:
      • decode

        public static java.lang.String decode​(java.lang.String s)
      • decodeIfNeeded

        public static java.lang.String decodeIfNeeded​(java.lang.String s)
        Unfortunately we have code that mindlessly decoded URLs (FileDownloadWindow) which borked (in the case I discovered) magnet uris with encoded parameters (e.g. the &tr= parameter) - doing so screws stuff up later if, for example, the parameter included an encoded '&'
        Parameters:
        s -
        Returns:
      • escapeXML

        public static java.lang.String escapeXML​(java.lang.String s)
      • unescapeXML

        public static java.lang.String unescapeXML​(java.lang.String s)
      • convertIPV6Host

        public static java.lang.String convertIPV6Host​(java.lang.String host)
      • expandIPV6Host

        public static java.lang.String expandIPV6Host​(java.lang.String host)
      • resolveIPv6Host

        public static java.lang.String resolveIPv6Host​(java.lang.String url)
      • resolveIPv4Host

        public static java.lang.String resolveIPv4Host​(java.lang.String url)
      • connectWithTimeout

        public static void connectWithTimeout​(java.net.URLConnection connection,
                                              long connect_timeout)
                                       throws java.io.IOException
        Throws:
        java.io.IOException
      • connectWithTimeouts

        public static void connectWithTimeouts​(java.net.URLConnection connection,
                                               long connect_timeout,
                                               long read_timeout)
                                        throws java.io.IOException
        Throws:
        java.io.IOException
      • setBrowserHeaders

        public static void setBrowserHeaders​(ResourceDownloader rd,
                                             java.lang.String referer)
      • setBrowserHeaders

        public static void setBrowserHeaders​(ResourceDownloader rd,
                                             java.lang.String encoded_headers,
                                             java.lang.String referer)
      • setBrowserHeaders

        public static void setBrowserHeaders​(ResourceUploader ru,
                                             java.lang.String encoded_headers,
                                             java.lang.String referer)
      • setBrowserHeaders

        public static void setBrowserHeaders​(java.net.URLConnection connection,
                                             java.lang.String referer)
      • setBrowserHeaders

        public static void setBrowserHeaders​(java.net.URLConnection connection,
                                             java.lang.String encoded_headers,
                                             java.lang.String referer)
      • getBrowserHeaders

        public static java.util.Map getBrowserHeaders​(java.lang.String referer)
      • getBrowserHeadersToUse

        private static java.lang.String getBrowserHeadersToUse​(java.lang.String encoded_headers)
      • queryHasParameter

        public static boolean queryHasParameter​(java.lang.String query_string,
                                                java.lang.String param_name,
                                                boolean case_sensitive)
      • containsPasskey

        public static boolean containsPasskey​(java.net.URL url)
      • setPort

        public static java.net.URL setPort​(java.net.URL u,
                                           int port)
      • setHost

        public static java.net.URL setHost​(java.net.URL u,
                                           java.lang.String host)
      • setProtocol

        public static java.net.URL setProtocol​(java.net.URL u,
                                               java.lang.String protocol)
      • getBaseURL

        public static java.net.URL getBaseURL​(java.net.URL u)
      • getCanonicalString

        public static java.lang.String getCanonicalString​(java.lang.String str)
      • getCanonicalString

        public static java.lang.String getCanonicalString​(java.net.URL url)
      • getIPV4Fallback

        public static java.net.URL getIPV4Fallback​(java.net.URL url)
        Returns an explicit IPv4 url if the supplied one has both IPv6 and IPv4 addresses
        Parameters:
        url -
        Returns:
      • getContentLength

        public static long getContentLength​(java.net.URLConnection con)
      • SSLSocketSNIHack

        public static boolean SSLSocketSNIHack​(java.lang.String host_name,
                                               javax.net.ssl.SSLSocket socket)
      • DHHackIt

        public static javax.net.ssl.SSLSocketFactory DHHackIt​(javax.net.ssl.SSLSocketFactory factory)
      • HTTPSURLConnectionSNIHack

        public static void HTTPSURLConnectionSNIHack​(java.lang.String host_name,
                                                     javax.net.ssl.HttpsURLConnection con)
      • DHHackIt

        public static void DHHackIt​(javax.net.ssl.HttpsURLConnection ssl_con)
      • connectSocketAndWrite

        public static java.net.Socket connectSocketAndWrite​(boolean is_ssl,
                                                            java.lang.String target_host,
                                                            int target_port,
                                                            byte[] bytes,
                                                            int connect_timeout,
                                                            int read_timeout)
                                                     throws java.lang.Exception
        Throws:
        java.lang.Exception
      • connectSocketAndWrite

        public static java.net.Socket connectSocketAndWrite​(boolean is_ssl,
                                                            java.lang.String target_host,
                                                            int target_port,
                                                            byte[] bytes,
                                                            int connect_timeout,
                                                            int read_timeout,
                                                            boolean unconnected_socket_hack)
                                                     throws java.lang.Exception
        Throws:
        java.lang.Exception