How to check if a specified key exists in a given S3 bucket using Java

Question

I would like to check if a key exists in a given bucket using Java  I looked at the API but there aren t any methods that are useful  I tried to use getObject but it threw an exception

User · Answer

Update   It seems there s a new API to check just that  See another answer in this page  https   stackoverflow com a 36653034 435605  Original post   Use errorCode equals  NoSuchKey    try       AmazonS3 s3   new AmazonS3Client new ClasspathPropertiesFileCredentialsProvider         String bucketName   getBucketName        s3 createBucket bucketName       S3Object object   s3 getObject bucketName  getKey       catch  AmazonServiceException e        String errorCode   e getErrorCode        if   errorCode equals  NoSuchKey              throw e            Logger getLogger getClass    debug  No such key      e       Note about the exception  I know exceptions should not be used for flow control  The problem is that Amazon didn t provide any api to check this flow - just documentation about the exception

User · Answer

There is an easy way to do it using jetS3t API s isObjectInBucket   method   Sample code   ProviderCredentials awsCredentials   new AWSCredentials                  awsaccessKey                  awsSecretAcessKey               REST implementation of S3Service         RestS3Service restService   new RestS3Service awsCredentials               check whether file exists in bucket         if  restService isObjectInBucket bucket  objectKey                    your logic

User · Answer

Break your path into bucket and object   Testing the bucket using the method doesBucketExist  Testing the object using the size of the listing  0 in case not exist   So this code will do   String bucket        String objectInBucket        AmazonS3 s3   new AmazonS3Client       return s3 doesBucketExist bucket           amp  amp   s3 listObjects bucket  objectInBucket  getObjectSummaries   isEmpty

User · Answer

There s now a doesObjectExist method in the official Java API   Enjoy

User · Answer

The other answers are for AWS SDK v1  Here is a method for AWS SDK v2  currently 2 3 9    Note that getObjectMetadata and doesObjectExist methods are not currently in the v2 SDK  So those are no longer options  We are forced to use either getObject or listObjects   listObjects calls are currently 12 5 times more expensive to make than getObject  But AWS also charges for any data downloaded  which raises the price of getObject if the file exists  As long as the file is very unlikely to exist  for example  you have generated a new UUID key randomly and just need to double-check that it isn t taken  then calling getObject is significantly cheaper by my calculation   Just to be on the safe side though  I added a range   specification to ask AWS to only send a few bytes of the file  As far as I know the SDK will always respect this and not charge you for downloading the whole file  But I haven t verified that so rely on that behavior at your own risk   Also  I m not sure what how range behaves if the S3 object is 0 bytes long        private boolean sanityCheckNewS3Key String bucket  String key             ResponseInputStream lt GetObjectResponse gt  resp   null          try               resp   s3client getObject GetObjectRequest builder                    bucket bucket                   key key                   range  bytes 0-3                    build                       catch  NoSuchKeyException e                return false                    catch  AwsServiceException se                throw se                    finally               if  resp    null                    try                       resp close                      catch  IOException e                        log warn  Exception while attempting to close S3 input stream   e                                                     return true            Note  this code assumes s3Client and log are declared and initialized elsewhere  Method returns a boolean  but can throw exceptions

User · Answer

Alternatively you can use Minio-Java client library  its Open Source and compatible with AWS S3 API   You can use Minio-Java StatObject java examples for the same     import io minio MinioClient  import io minio errors MinioException   import java io InputStream  import java io IOException  import java security NoSuchAlgorithmException  import java security InvalidKeyException   import org xmlpull v1 XmlPullParserException    public class GetObject     public static void main String   args      throws NoSuchAlgorithmException  IOException  InvalidKeyException  XmlPullParserException  MinioException          Note  YOUR-ACCESSKEYID  YOUR-SECRETACCESSKEY and my-bucketname are        dummy values  please replace them with original values         Set s3 endpoint  region is calculated automatically     MinioClient s3Client   new MinioClient  https   s3 amazonaws com    YOUR-ACCESSKEYID    YOUR-SECRETACCESSKEY        InputStream stream   s3Client getObject  my-bucketname    my-objectname         byte   buf   new byte 16384       int bytesRead      while   bytesRead   stream read buf  0  buf length      0          System out println new String buf  0  bytesRead               stream close            I hope it helps   Disclaimer   I work for Minio

User · Answer

Using Object isting  Java function to check if specified key exist in AWS S3   boolean isExist String key                ObjectListing objects   amazonS3 listObjects new ListObjectsRequest   withBucketName bucketName  withPrefix key             for  S3ObjectSummary objectSummary   objects getObjectSummaries                          if  objectSummary getKey   equals key                                 return true                                   return false

User · Answer

In Amazon Java SDK 1 10   you can use getStatusCode   to get the status code of the HTTP response  which will be 404 if the object does not exist   import com amazonaws services s3 AmazonS3  import com amazonaws services s3 model AmazonS3Exception  import org apache http HttpStatus   try       AmazonS3 s3   new AmazonS3Client        ObjectMetadata object   s3 getObjectMetadata  my-bucket    my-client      catch  AmazonS3Exception e        if  e getStatusCode      HttpStatus SC NOT FOUND               bucket key does not exist        else           throw e            getObjectMetadata   consumes fewer resources  and the response doesn t need to be closed like getObject        In previous versions  you can use getErrorCode   and check for the appropriate string  depends on the version

User · Answer

The right way to do it in SDK V2  without the overload of actually getting the object  is to use S3Client headObject  Officially backed by AWS Change Log  Example code  public boolean exists String bucket  String key        try           HeadObjectResponse headResponse   client                  headObject HeadObjectRequest builder   bucket bucket  key key  build             return true        catch  NoSuchKeyException e            return false

User · Answer

Use ListObjectsRequest setting Prefix as your key    NET code    public bool Exists string key                 using  Amazon S3 AmazonS3Client client    Amazon S3 AmazonS3Client Amazon AWSClientFactory CreateAmazonS3Client m accessKey  m accessSecret                         ListObjectsRequest request   new ListObjectsRequest                request BucketName   m bucketName              request Prefix   key              using  ListObjectsResponse response   client ListObjects request                                  foreach  S3Object o in response S3Objects                                        if  o Key    key                           return true                                    return false

User · Answer

For PHP  I know the question is Java  but Google brought me here   you can use stream wrappers and file exists   bucket    MyBucket    key    MyKey    s3   Aws S3 S3Client- gt factory          s3- gt registerStreamWrapper     keyExists   file exists  s3    bucket  key

User · Answer

Use the jets3t library  Its a lot more easier and robust than the AWS sdk  Using this library you can call  s3service getObjectDetails    This will check and retrieve only the details of the object  not the contents  of the object  It will throw a 404 if the object is missing  So you can catch that exception and deal with it in your app   But in order for this to work  you will need to have ListBucket access for the user on that bucket  Just GetObject  access will not work  The reason being  Amazon will prevent you from checking for the presence of the key if you dont have ListBucket access  Just knowing whether a key is present or not  will also suffice for malicious users in some cases  Hence unless they have ListBucket access they will not be able to do so

User · Answer

I also faced this problem when I used String BaseFolder    quot 3patti Logs quot    S3Object object   s3client getObject bucketName  BaseFolder      I got error key not found When I hit and try String BaseFolder    quot 3patti Logs quot    S3Object object   s3client getObject bucketName  BaseFolder  quot   quot     it worked   this code is working with 1 9 jar otherwise update to 1 11 and use doesObjectExist as said above

User · Answer

Using the AWS SDK use the getObjectMetadata method   The method will throw an AmazonServiceException if the key doesn t exist   private AmazonS3 s3      public boolean exists String path  String name        try           s3 getObjectMetadata bucket  getS3Path path    name          catch AmazonServiceException e            return false            return true

User · Answer

This java code checks if the key  file  exists in s3 bucket   public static boolean isExistS3 String accessKey  String secretKey  String bucketName  String file            Amazon-s3 credentials     AWSCredentials myCredentials   new BasicAWSCredentials accessKey  secretKey        AmazonS3Client s3Client   new AmazonS3Client myCredentials         ObjectListing objects   s3Client listObjects new ListObjectsRequest   withBucketName bucketName  withPrefix file         for  S3ObjectSummary objectSummary  objects getObjectSummaries              if  objectSummary getKey   equals file                 return true                      return false

User · Answer

As others have mentioned  for the AWS S3 Java SDK 2 10  you can use the HeadObjectRequest object to check if there is a file in your S3 bucket  This will act like a GET request without actually getting the file  Example code since others haven t actually added any code above  public boolean existsOnS3    throws Exception       try          S3Client s3Client   S3Client builder    credentialsProvider       build            HeadObjectRequest headObjectRequest   HeadObjectRequest builder    bucket   quot my-bucket quot   key   quot key to file house pdf quot   build            HeadObjectResponse headObjectResponse   s3Client headObject  headObjectRequest          return headObjectResponse sdkHttpResponse    isSuccessful                 catch  NoSuchKeyException e            Log exception for debugging       return false

[java] How to check if a specified key exists in a given S3 bucket using Java

Examples related to java

Examples related to amazon-web-services

Examples related to amazon-s3

Examples related to aws-sdk