Monday, 5 October 2020

How to append content to a DBFS file using python spark


You can read and write to DBFS files using 'dbutils'

Lets see one example

dbutils.fs.put("dbfs:///mnt/sample.txt", "sample content")

Above command helps to write "sample content" to 'dbfs:///mnt/sample.txt'

Now you have the file in your DBFS location. Lets see how to append content to this file.


In order to do that, you can open the file in append mode.



with  open("/dbfs/mnt/sample.txt""a") as f:
  f.write("append values")

Now your appended file is ready!!!


 

13 comments:

  1. i am not able to do that in databricks it gives me below error

    1 with open("/dbfs/mnt/vdmdev/TempFiles/201905.log", "a") as f:
    ----> 2 f.write("append values")

    OSError: [Errno 95] Operation not supported

    ReplyDelete
    Replies
    1. I am also facing same issue

      Delete
    2. I also got the same issue. Any help is appreciated.

      Delete
    3. Can you check if 201905.log got created on dbfs

      Delete
  2. "I visited your blog you have shared amazing information, i really like the information provided by you, You have done a great work. I hope you will share some more information regarding full movies online. I appreciate your work.
    Thanks"

    Web Design Gloucester
    Web Design Company Cheltenham
    SEO Agency Cheltenham

    ReplyDelete
  3. Find Ivy Bronx TV Cabinet, Black tv stand with led lights, Can Accommodate 55-Inch TV Screen,Black reviews & recommendations from people you can trust.

    ReplyDelete
  4. Your content is informative, yet I feel there's more to explore. Could you consider providing additional resources or references for further reading
    ''thank you''
    Apartment for sale at Thimphu

    ReplyDelete
  5. Your writing style is engaging, and the insights shared are thought-provoking. Looking forward to more content.

    ReplyDelete
  6. Wow I am here again not by accident because I choose to be. I am really loving this content.. I have even recommended this to my friends.

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete