Package ucar.unidata.util

Class IOUtil

java.lang.Object
ucar.unidata.util.IOUtil
public class IOUtil extends Object
A set of io related utilities
Version:
$Revision: 1.52 $
Author:
IDV development group.

  • Field Details

    • MAX_REDIRECTS

      public static final int MAX_REDIRECTS
      Maximum number of HTTP redirects to follow.
      See Also:

    • HTTP_REDIRECT_STATUSES

      public static final int[] HTTP_REDIRECT_STATUSES
      Elements within this array are considered "valid redirect" status codes.

    • HTTP_TIMEOUT

      public static final int HTTP_TIMEOUT
      HTTP connection timeout (milliseconds).
      See Also:

    • debug

      public static boolean debug
      debug flag

  • Constructor Details

    • IOUtil

      public IOUtil()
      Default constructor; does nothing

  • Method Details

    • hasSuffix

      public static boolean hasSuffix(String fileOrUrl, String suffix)
      Does the given file or url have the given suffix
      Parameters:
      fileOrUrl - The name of the file or url
      suffix - The suffix
      Returns:
      Does the fileOrUrl have the suffix

    • getMostRecentFile

      public static File getMostRecentFile(File dir)
      Find the youngest file in the given directory.
      Parameters:
      dir - The directory to search in.
      Returns:
      The most recent file (or null if none found).

    • getMostRecentFile

      public static File getMostRecentFile(File dir, FileFilter filter)
      Find the youngest file in the given directory that matches the given FileFilter.
      Parameters:
      dir - The directory to search in.
      filter - The FileFilter to be used to limit what files we look at (may be null).
      Returns:
      The most recent file (or null if none found).

    • sortFilesOnAge

      public static File[] sortFilesOnAge(File directory, FileFilter filter, boolean youngestFirst)
      Sort the files contained by the given directory and that (if non-null) match the given filter.
      Parameters:
      directory - The directory
      filter - The filter
      youngestFirst - Ascending or descending
      Returns:
      The sorted files

    • getNormalFiles

      public static File[] getNormalFiles(File[] files)
      Return an array of the file Files. That is, the ones where File.isFile is true
      Parameters:
      files - Array of files
      Returns:
      The files

    • sortFilesOnAge

      public static File[] sortFilesOnAge(File[] files, boolean youngestFirst)
      Sort the given files
      Parameters:
      files - The files
      youngestFirst - Ascending or descending
      Returns:
      Just return the given array

    • sortFilesOnAge

      public static IOUtil.FileWrapper[] sortFilesOnAge(IOUtil.FileWrapper[] files)
      _more_
      Parameters:
      files - _more_
      Returns:
      _more_

    • deleteFiles

      public static List<File> deleteFiles(List<File> files)
      This deletes the files in the given list. It will return a list of those files where file.delete was not successful
      Parameters:
      files - The files to delete
      Returns:
      List of unsuccessfully deleted files

    • findFilesToScour

      public static List<File> findFilesToScour(File dir, double hours, long maxBytes)
      This finds and returns a list files to scour
      Parameters:
      dir - _more_
      hours - _more_
      maxBytes - _more_
      Returns:
      _more_

    • millisToMinutes

      public static double millisToMinutes(double millis)
      _more_
      Parameters:
      millis - _more_
      Returns:
      _more_

    • sortFilesOnName

      public static File[] sortFilesOnName(File[] files)
      _more_
      Parameters:
      files - _more_
      Returns:
      _more_

    • sortFilesOnName

      public static File[] sortFilesOnName(File[] files, boolean descending)
      _more_
      Parameters:
      files - _more_
      descending - _more_
      Returns:
      _more_

    • toFiles

      public static File[] toFiles(List files)
      Convert the toString value of the objects in the given files list to an array of File-s
      Parameters:
      files - List of files
      Returns:
      array of files

    • wrapFilter

      public static FileFilter wrapFilter(FileFilter filter)
      Create a javaio FileFilter from the filechooser package file filter.
      Parameters:
      filter - The filechooser file filter
      Returns:
      The javaio FileFilter.

    • getMostRecentFile

      public static File getMostRecentFile(File dir, FileFilter filter)
      Find the youngest file in the given directory that matches the given FileFilter.
      Parameters:
      dir - The directory to search in.
      filter - The filter to be used to limit what files we look at (may be null).
      Returns:
      The most recent file (or null if none found).

    • writeTo

      public static long writeTo(InputStream from, OutputStream to) throws IOException
      Copy the input stream to the output stream
      Parameters:
      from - input stream
      to - output
      Returns:
      How may bytes were written
      Throws:
      IOException - On badness

    • write

      public static void write(OutputStream to, String s) throws Exception
      Write the bytes held by the iven string to the outputstream
      Parameters:
      to - stream to write to
      s - string to write
      Throws:
      Exception - on badness

    • readContents

      public static String readContents(URL url) throws Exception
      _more_
      Parameters:
      url - _more_
      Returns:
      _more_
      Throws:
      Exception - _more_

    • getUrlConnection

      public static URLConnection getUrlConnection(String url) throws IOException
      Attempt to create a URLConnection to the given url.

      This method will follow redirects, and will use HTTP_TIMEOUT, MAX_REDIRECTS, and HTTP_REDIRECT_STATUSES as default values to pass to getUrlConnection(String, boolean, boolean, int, int, int[]).

      Parameters:
      url - Request URL.
      Returns:
      Connection to url.
      Throws:
      IOException - if there were any I/O errors while trying to connect.
      See Also:

    • getUrlConnection

      public static URLConnection getUrlConnection(String url, boolean allowUserInteraction, boolean followRedirects, int timeout, int maxRedirects, int[] redirectStatuses) throws IOException
      Attempt to create a URLConnection to the given url.

      If followRedirects is true, the maximum number of redirects is controlled via MAX_REDIRECTS. If followRedirects is false, the method, predictably, will not follow any redirects.

      Parameters:
      url - Request URL.
      allowUserInteraction - Passed to URLConnection.setAllowUserInteraction(boolean).
      followRedirects - Whether or not HTTP redirect codes should be followed.
      timeout - HTTP connection timeout (milliseconds).
      maxRedirects - Maximum number of redirects to follow.
      redirectStatuses - Status codes that are considered HTTP redirects.
      Returns:
      Connection to url.
      Throws:
      IOException - if there were any I/O errors while trying to connect.

    • writeTo

      public static long writeTo(URL from, File file, Object loadId) throws IOException
      Write to the file from the URL stream
      Parameters:
      from - URL for input
      file - file for output
      loadId - A JobManager loadId that, if set, can be used to stop the load
      Returns:
      number of bytes written
      Throws:
      IOException - problem writing to file.

    • writeTo

      public static long writeTo(InputStream from, OutputStream to, Object loadId, long length) throws IOException
      Copy the input stream to the output stream
      Parameters:
      from - input stream
      to - output
      loadId - A JobManager loadId that, if set, can be used to stop the load
      length - number of bytes to write
      Returns:
      How may bytes were written
      Throws:
      IOException - On badness

    • writeTo

      public static List writeTo(List urls, String prefix, String suffix) throws IOException
      Copy the files pointed to by the urls list to the directory/file pointed to by prefx. We copy prefix0.suffix,prefix1.suffix, ..., prefixN.suffix
      Parameters:
      urls - List of urls to copy
      prefix - file directory prefix
      suffix - suffix
      Returns:
      List of new files
      Throws:
      IOException - On badness

    • writeTo

      public static List writeTo(List urls, String prefix, String suffix, Object loadId) throws IOException
      Copy the files pointed to by the urls list to the directory/file pointed to by prefx. We copy prefix0.suffix,prefix1.suffix, ..., prefixN.suffix
      Parameters:
      urls - List of urls to copy
      prefix - file directory prefix
      suffix - suffix
      loadId - JobManager loadId
      Returns:
      List of new files
      Throws:
      IOException - On badness

    • writeTo

      public static List writeTo(List urls, String prefix, List suffixes, Object loadId) throws IOException
      _more_
      Parameters:
      urls - _more_
      prefix - _more_
      suffixes - _more_
      loadId - _more_
      Returns:
      _more_
      Throws:
      IOException - _more_

    • getListing

      public static List<String> getListing(String path, Class c)
      Lists the contents in the given file directory or java resource directory
      Parameters:
      path - may be a file directory path or a java resource directory
      c - base class to use if its java resources
      Returns:
      List of sub files and directories

    • getFileTail

      public static String getFileTail(String f)
      Gets the file name, removing any leading directory paths.
      Parameters:
      f - The file path.
      Returns:
      The file name.

    • getFileRoot

      public static String getFileRoot(String f)
      Excise the filename from the given path and return the root.
      Parameters:
      f - The file path.
      Returns:
      The file name.

    • stripExtension

      public static String stripExtension(String f)
      Remove any file extension from the given file name.
      Parameters:
      f - The file path.
      Returns:
      The file name without the extension.

    • cleanFileName

      public static String cleanFileName(String name)
      Remove illegal characters in the given filename
      Parameters:
      name - The filename to be cleaned up
      Returns:
      The cleaned up filename

    • getFileExtension

      public static String getFileExtension(String f)
      Return the file extension from the given file (including the ".").
      Parameters:
      f - The file path.
      Returns:
      The file extension or an empty string if none found.

    • writeFile

      public static void writeFile(String filename, String contents) throws FileNotFoundException, IOException
      Write out a file to the filename specified.
      Parameters:
      filename - filename to write to
      contents - file contents
      Throws:
      FileNotFoundException - if the file does not exist
      IOException - if there is a problem writing

    • writeFile

      public static void writeFile(File filename, String contents) throws FileNotFoundException, IOException
      Write out a file to the File specified.
      Parameters:
      filename - File to write to
      contents - file contents
      Throws:
      FileNotFoundException - if the file does not exist
      IOException - if there is a problem writing

    • writeBytes

      public static void writeBytes(File filename, byte[] contents) throws FileNotFoundException, IOException
      Write out a file to the File specified.
      Parameters:
      filename - File to write to
      contents - file contents
      Throws:
      FileNotFoundException - if the file does not exist
      IOException - if there is a problem writing

    • moveFile

      public static void moveFile(File from, File to) throws FileNotFoundException, IOException
      Move the from file to the to file
      Parameters:
      from - File to move
      to - The destination
      Throws:
      FileNotFoundException - When we cannot find the file
      IOException - When something untoward happens

    • copyFile

      public static void copyFile(File from, File to) throws FileNotFoundException, IOException
      Copy from file to to file
      Parameters:
      from - File to copy
      to - The destination
      Throws:
      FileNotFoundException - When we cannot find the file
      IOException - When something untoward happens

    • copyFile

      public static void copyFile(InputStream fis, File to) throws FileNotFoundException, IOException
      Copy from inputstream to to file
      Parameters:
      fis - source
      to - The destination
      Throws:
      FileNotFoundException - When we cannot find the file
      IOException - When something untoward happens

    • isTextFile

      public static boolean isTextFile(String filename)
      Determine if the given filename is a text file. i.e., it ends with .txt or .text
      Parameters:
      filename - The filename to check.
      Returns:
      Is the filename a text file.

    • isImageFile

      public static boolean isImageFile(String filename)
      Determine if the given filename is an image file (e.g., ends with .gif, .jpg, .jpeg, .png)
      Parameters:
      filename - The filename to check.
      Returns:
      Is the filename an image file.

    • isHtmlFile

      public static boolean isHtmlFile(String filenameOrUrl)
      Determine if the given filename is an html file. i.e., it ends with .htm or .html or if it does not have a file extension and begins with http:
      Parameters:
      filenameOrUrl - The filename to check.
      Returns:
      Is the filename an html file.

    • isHtmlSuffix

      public static boolean isHtmlSuffix(String url)
      Does the given url end with an html suffix.
      Parameters:
      url - The url
      Returns:
      Ends with html, htm, or shtml

    • isHttpProtocol

      public static boolean isHttpProtocol(String url)
      Is the given url an http protocol.
      Parameters:
      url - The url
      Returns:
      Starts with http or https

    • getInputStream

      public static InputStream getInputStream(String filename) throws FileNotFoundException, IOException
      Get an input stream for the filename
      Parameters:
      filename - name of file
      Returns:
      corresponding input stream
      Throws:
      FileNotFoundException - couldn't find the file
      IOException - problem opening stream

    • getInputStream

      public static InputStream getInputStream(String filename, Class origin) throws FileNotFoundException, IOException
      Get an input stream for the filename
      Parameters:
      filename - name of file
      origin - relative origin point for file location
      Returns:
      corresponding input stream
      Throws:
      FileNotFoundException - couldn't find the file
      IOException - problem opening stream

    • openConnection

      protected static URLConnection openConnection(URL url) throws IOException
      Throws:
      IOException

    • setUserAccountManager

      public static void setUserAccountManager(IOUtil.UserAccountManager manager)
      _more_
      Parameters:
      manager - _more_

    • getInputStreamOLDWAY

      public static InputStream getInputStreamOLDWAY(String filename, Class origin) throws FileNotFoundException, IOException
      Get an input stream for the filename
      Parameters:
      filename - name of file
      origin - relative origin point for file location
      Returns:
      corresponding input stream
      Throws:
      FileNotFoundException - couldn't find the file
      IOException - problem opening stream

    • getURL

      public static URL getURL(String filename, Class origin) throws FileNotFoundException, IOException
      Get an input stream for the filename
      Parameters:
      filename - name of file
      origin - relative origin point for file location
      Returns:
      corresponding input stream
      Throws:
      FileNotFoundException - couldn't find the file
      IOException - problem opening stream

    • readContents

      public static String readContents(File file) throws FileNotFoundException, IOException
      Read the contents of a File. Used for reading text type files (XML, HTML, etc)
      Parameters:
      file - file to read.
      Returns:
      contents as a String
      Throws:
      FileNotFoundException - couldn't find the file
      IOException - problem opening stream

    • isADescendent

      public static boolean isADescendent(File parent, File child)
      _more_
      Parameters:
      parent - _more_
      child - _more_
      Returns:
      _more_

    • readContents

      public static String readContents(String contentName, String dflt)
      Return the String contents of the specified contentName. If the read fails (for whatever reason) then return the dflt parameter
      Parameters:
      contentName - URL or filename
      dflt - default to return if a problem
      Returns:
      contents or default value

    • readContents

      public static String readContents(String contentName, Class origin, String dflt)
      Return the String contents of the specified contentName. If the read fails (for whatever reason) then return the dflt parameter
      Parameters:
      contentName - URL or filename
      origin - origin class
      dflt - default to return if a problem
      Returns:
      contents or default value

    • readContents

      public static String readContents(String contentName) throws FileNotFoundException, IOException
      Return the String contents of the specified contentName.
      Parameters:
      contentName - can either be a URL, a filename or a resource.
      Returns:
      contents or null if there is a problem.
      Throws:
      FileNotFoundException - couldn't find the file
      IOException - problem reading contents

    • isRelativePath

      public static boolean isRelativePath(String path)
      Is the given path relative
      Parameters:
      path - file path
      Returns:
      is relative

    • readContents

      public static String readContents(String contentName, Class origin) throws FileNotFoundException, IOException
      Return the String contents of the specified contentName. contentName can either be a URL, a filename or a resource.
      Parameters:
      contentName - can either be a URL, a filename or a resource.
      origin - relative origin for path to file
      Returns:
      contents or null if there is a problem.
      Throws:
      FileNotFoundException - couldn't find the file
      IOException - problem reading contents

    • getFinalURL

      public static String getFinalURL(String urlStr)
      Return the String contents of the final url string.
      Parameters:
      urlStr - can be a URL .
      Returns:
      contents or null if there is a problem.

    • readBytesAndCache

      public static byte[] readBytesAndCache(String contentName, String cacheGroup) throws FileNotFoundException, IOException
      See if the content is in the perma-cache. If it is then return it. Else read it (e.g., from a url) and cache it.
      Parameters:
      contentName - url or filename
      cacheGroup - Cache group
      Returns:
      Bytes read
      Throws:
      FileNotFoundException - On badness
      IOException - On badness

    • isZipFile

      public static boolean isZipFile(String file)
      _more_
      Parameters:
      file - _more_
      Returns:
      _more_

    • readBytesAndCache

      public static byte[] readBytesAndCache(String contentName, String cacheGroup, boolean unzipIfNeeded) throws FileNotFoundException, IOException
      See if the content is in the perma-cache. If it is then return it. Else read it (e.g., from a url) and cache it.
      Parameters:
      contentName - url or filename
      cacheGroup - Cache group
      unzipIfNeeded - IF true and if the url is a zip file then unzip it
      Returns:
      Bytes read
      Throws:
      FileNotFoundException - On badness
      IOException - On badness

    • readContents

      public static String readContents(InputStream is) throws IOException
      Read in the bytes from the given InputStream and construct and return a String. Closes the InputStream argument.
      Parameters:
      is - InputStream to read from
      Returns:
      contents as a String
      Throws:
      IOException - problem reading contents

    • readInputStream

      public static String readInputStream(InputStream is) throws IOException
      Read in the bytes from the given InputStream and construct and return a String. Closes the InputStream argument.
      Parameters:
      is - InputStream to read from
      Returns:
      contents as a String
      Throws:
      IOException - problem reading contents

    • readBytes

      public static byte[] readBytes(InputStream is) throws IOException
      Read in the bytes from the given InputStream Closes the InputStream argument.
      Parameters:
      is - InputStream to read from
      Returns:
      bytes read
      Throws:
      IOException - problem reading contents

    • readBytes

      public static byte[] readBytes(InputStream is, Object loadId) throws IOException
      Read in the bytes from the given InputStream Closes the InputStream argument. The globalTimestamp and myTimestamp parameters, if non-null, allow calling routines to abort the read. globalTimestamp, if non-null, is a array of size one that holds a virtual timestamp. If at anytime during the read of the bytes the value in globalTimestamp is different then the value given for myTimestamp then the read is aborted and null is returned.
      Parameters:
      is - InputStream to read from
      loadId - Job manager load id
      Returns:
      bytes read
      Throws:
      IOException - problem reading contents

    • close

      public static void close(InputStream inputStream)
      _more_
      Parameters:
      inputStream - _more_

    • close

      public static void close(OutputStream outputStream)
      _more_
      Parameters:
      outputStream - _more_

    • readBytes

      public static byte[] readBytes(InputStream is, Object loadId, boolean closeIt) throws IOException
      Read the bytes in the given input stream.
      Parameters:
      is - The input stream
      loadId - If non-null check with the JobManager if we should continue
      closeIt - If true then close the input stream
      Returns:
      The bytes
      Throws:
      IOException - On badness

    • joinDir

      public static String joinDir(String f1, String f2)
      Return the String representation of the given filename joined to the given directory.
      Parameters:
      f1 - directory path
      f2 - filename
      Returns:
      concatenated String with the appropriate file separator

    • joinDir

      public static String joinDir(File f1, String filename)
      Return the String representation of the given filename joined to the given directory f1.
      Parameters:
      f1 - directory path
      filename - filename
      Returns:
      concatenated String with the appropriate file separator

    • joinDirs

      public static String joinDirs(String... f)
      Return the String representation of the given path separated by args.
      Parameters:
      f - directory path vararg
      Returns:
      concatenated String with the appropriate file separator

    • makeDir

      public static String makeDir(String path)
      If the directory defined in the given argument path does not exist then make it.
      Parameters:
      path - directory to make
      Returns:
      the directory path

    • makeDir

      public static String makeDir(File f)
      If the directory defined in the given argument f does not exist then make it.
      Parameters:
      f - directory as a file
      Returns:
      directory path

    • makeDirRecursive

      public static final void makeDirRecursive(File f)
      Make the directory. This will recurse and make the directory tree if needed
      Parameters:
      f - directory to make

    • getFiles

      public static List<File> getFiles(File dir, boolean recurse)
      Recursively descend (if recurse is true) through the given directory and return a list of all files
      Parameters:
      dir - The directory to look at
      recurse - Do we recurse
      Returns:
      List of files

    • getFiles

      public static List<File> getFiles(List files, File dir, boolean recurse)
      Recursively descend (if recurse is true) through the given directory and return a list of all files
      Parameters:
      files - The list of files to add to
      dir - The directory to look at
      recurse - Do we recurse
      Returns:
      List of files

    • getFiles

      public static List<File> getFiles(List files, File dir, boolean recurse, PatternFileFilter filter)
      Recursively descend (if recurse is true) through the given directory and return a list of all files
      Parameters:
      files - The list of files to add to
      dir - The directory to look at
      recurse - Do we recurse
      filter - If non-null then use this to find files
      Returns:
      List of files

    • walkDirectory

      public static boolean walkDirectory(File dir, IOUtil.FileViewer fileViewer) throws Exception
      Walk the dir tree with the given file viewer
      Parameters:
      dir - dir
      fileViewer - viewer
      Returns:
      should continue
      Throws:
      Exception - on badness_

    • walkDirectory

      public static boolean walkDirectory(File dir, IOUtil.FileViewer fileViewer, int level) throws Exception
      Walk the dir tree with the given file viewer
      Parameters:
      dir - dir
      fileViewer - viewer
      level - tree depth
      Returns:
      should continue
      Throws:
      Exception - on badness_

    • getDirectories

      public static List<File> getDirectories(File dir, boolean recurse)
      Recursively descend (if recurse is true) through the given directory and return a list of all subdirectories.
      Parameters:
      dir - The directory to look at
      recurse - Do we recurse
      Returns:
      List of subdirs (File)

    • getDirectories

      public static List<File> getDirectories(List dirs, boolean recurse)
      Recursively descend (if recurse is true) through the given directories and return a list of all subdirectories.
      Parameters:
      dirs - List of directories to look at
      recurse - Do we recurse
      Returns:
      List of subdirs (File)

    • pruneIfEmpty

      public static void pruneIfEmpty(File root)
      This will recursively prune empty subdirectories of the given root directory. It will delete the given root directory if it is empty.
      Parameters:
      root - Directory to prune

    • pruneEmptyDirectories

      public static void pruneEmptyDirectories(File root)
      This will recursively prune empty subdirectories of the given root directory. It will NOT delete the given root directory if it is empty.
      Parameters:
      root - Directory to prune

    • deleteDirectory

      public static void deleteDirectory(File root)
      This will recursively delete all contents under the given directory.
      Parameters:
      root - Directory to delete

    • wait

      public static void wait(File directory, String filePattern, double sleepSeconds)
      Wait until there are new files in the given directory. If filePattern is non null then use that (regular expression) to match the files.
      Parameters:
      directory - The directory
      filePattern - The pattern
      sleepSeconds - Sleep this number of seconds between each check

    • wait

      public static void wait(List files, double sleepSeconds)
      Wait until one or more of the files in the files list (File) has changed. If the files list is null or empty then just return.
      Parameters:
      files - List of File-s
      sleepSeconds - Seconds to sleep between checks.

    • readSerializedObject

      public static Object readSerializedObject(String file) throws Exception
      _more_
      Parameters:
      file - _more_
      Returns:
      _more_
      Throws:
      Exception - _more_

    • writeSerializedObject

      public static void writeSerializedObject(String file, Object o) throws Exception
      _more_
      Parameters:
      file - _more_
      o - _more_
      Throws:
      Exception - _more_

    • testio

      public static void testio() throws Exception
      _more_
      Throws:
      Exception - _more_

    • main

      public static void main(String[] args) throws Exception
      test mainp
      Parameters:
      args - cmd line args
      Throws:
      Exception - On badness

    • writeJarFile

      public static void writeJarFile(String filename, List files) throws IOException
      Write out the list of files to the jar file specified by filename
      Parameters:
      filename - jar file name
      files - list of files
      Throws:
      IOException - On badness

    • writeJarFile

      public static void writeJarFile(String filename, List files, String pathPrefix) throws IOException
      Write out the list of files to the jar file specified by filename
      Parameters:
      filename - jar file name
      files - list of files
      pathPrefix - If not null this is the prefx we add to the jar entry
      Throws:
      IOException - On badness

    • writeJarFile

      public static void writeJarFile(String filename, List files, String pathPrefix, boolean makeFilesUnique) throws IOException
      Write out the list of files to the jar file specified by filename
      Parameters:
      filename - jar file name
      files - list of files
      pathPrefix - If not null this is the prefx we add to the jar entry
      makeFilesUnique - If true then make all of the file names unique in case of conflict
      Throws:
      IOException - On badness

    • createChecksum

      public static byte[] createChecksum(String filename) throws Exception
      _more_
      Parameters:
      filename - _more_
      Returns:
      _more_
      Throws:
      Exception - _more_

    • getMD5Checksum

      public static String getMD5Checksum(String filename) throws Exception
      _more_
      Parameters:
      filename - _more_
      Returns:
      _more_
      Throws:
      Exception - _more_

    • getMd5

      public static String getMd5(String path) throws Exception
      _more_
      Parameters:
      path - _more_
      Returns:
      _more_
      Throws:
      Exception - _more_