Unable to write to binary file using pickle Python

So I'm having this problem where I'm trying to write a dictionary to a binary file and nothing happens. Very strange; no error message, nothing. It's like the program skips over the code, specifically this code

with open("database.dat", "wb") as handle:

        pickle.dump(d, handle)

        handle.close()

I have no problem loading from this database.dat file if it exists but this code doesn't seem to create a new binary file if it doesn't exist and it doesn't overwrite the file if it does exist. I've never had this problem so I'm perplexed...

To give an idea of where this fits in, this the function:

import pickle



def initialise_database(directory, file_name):



    try:

        with open("database.dat", "rb") as handle:

            d = pickle.load(handle) # THIS WORKS PERFECTLY

            handle.close()

            return d



    except FileNotFoundError:

        return update_database(directory, file_name)





def update_database(directory, file_name):

    import xlrd, os



    # Change directory

    os.chdir(directory)



    try:

        wb = xlrd.open_workbook(file_name)



        sheet = wb.sheet_by_name("FUNDS")



        rows, cols = sheet.nrows, sheet.ncols



        d = {}

        for i in range(rows - 1):

            if sheet.cell_value(i + 1, 0) != "":

                charity = cap(sheet.cell_value(i + 1, 0))



                area_of_work = 

                org = 

                funding = 0



                for x in range(cols):

                    if sheet.cell_value(0, x) == "Area of work":

                        if sheet.cell_value(i + 1, x) != "":

                            area_of_work.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass



                    elif sheet.cell_value(0, x) == "Organisation funded":

                        if sheet.cell_value(i + 1, x) != "":

                            org.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass





                    elif sheet.cell_value(0, x) == "How much funding provided":

                        funding = (str_to_int(sheet.cell_value(i + 1, x)))



                        if funding is False:

                            print("nCharity " + str(sheet.cell_value(i + 1, 0)) + ", number " + str(i + 2) + " has funding of value: "+ str(funding) + ", which is not compatible. Please find it and fix it in the the excel file. Then try again")

                            return False



                # Adds entry to dictionary

                d[charity] = [area_of_work, org, funding]



            else:

                pass



        # After the loop has finished, it should write the entire dictionary to a newly created file!



        with open("database.dat", "wb") as handle:

            pickle.dump(d, handle)

            handle.close()



        return d



    except FileNotFoundError:

        print("File could not be found.")

        print("Program terminating.nnPress the enter key to exit.")

I've taken a look at other questions and none seem to touch on the problem I'm facing here. Any help on how to fix it would be greatly appreciated!

Edit:
I'd also like to add that when I run the code and perform

print(initialise_database( #my_directory, #my_file )

It does print out my database, meaning that the update() function is being called, seeing as there is no such database.dat file anywhere

As proof that the file is not appearing, here is a screenshot. This is after full_script (which is where this code is taken from) has been ran.

Evidence there is no file

edited Jan 2 at 20:36

asked Jan 2 at 20:09

Tom_G_99

429

2

Why do you close handle? The withtakes care of that.

– progmatico
Jan 2 at 20:14

he has his return statement inside his with block. meaning it might not close, unless the with did close it.

– GeeTransit
Jan 2 at 20:18

@progmatico haha, thanks for pointing that out. I didn't know that. I'm still quite new to programming... I've taken it out but still no change :( idk why no binary file is being created. I double-checked all my directories and none have the file. Thus, it should create the file!

– Tom_G_99
Jan 2 at 20:20

have you checked in the directory the python file is in?

– GeeTransit
Jan 2 at 20:25

2

@GeeTransit Doesn't directly relate to the question, but with guarantees that the file closes even if there is a return statement.

– Tomothy32
Jan 2 at 20:28

|
show 7 more comments

with open("database.dat", "wb") as handle:

        pickle.dump(d, handle)

        handle.close()

To give an idea of where this fits in, this the function:

import pickle



def initialise_database(directory, file_name):



    try:

        with open("database.dat", "rb") as handle:

            d = pickle.load(handle) # THIS WORKS PERFECTLY

            handle.close()

            return d



    except FileNotFoundError:

        return update_database(directory, file_name)





def update_database(directory, file_name):

    import xlrd, os



    # Change directory

    os.chdir(directory)



    try:

        wb = xlrd.open_workbook(file_name)



        sheet = wb.sheet_by_name("FUNDS")



        rows, cols = sheet.nrows, sheet.ncols



        d = {}

        for i in range(rows - 1):

            if sheet.cell_value(i + 1, 0) != "":

                charity = cap(sheet.cell_value(i + 1, 0))



                area_of_work = 

                org = 

                funding = 0



                for x in range(cols):

                    if sheet.cell_value(0, x) == "Area of work":

                        if sheet.cell_value(i + 1, x) != "":

                            area_of_work.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass



                    elif sheet.cell_value(0, x) == "Organisation funded":

                        if sheet.cell_value(i + 1, x) != "":

                            org.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass





                    elif sheet.cell_value(0, x) == "How much funding provided":

                        funding = (str_to_int(sheet.cell_value(i + 1, x)))



                        if funding is False:

                            print("nCharity " + str(sheet.cell_value(i + 1, 0)) + ", number " + str(i + 2) + " has funding of value: "+ str(funding) + ", which is not compatible. Please find it and fix it in the the excel file. Then try again")

                            return False



                # Adds entry to dictionary

                d[charity] = [area_of_work, org, funding]



            else:

                pass



        # After the loop has finished, it should write the entire dictionary to a newly created file!



        with open("database.dat", "wb") as handle:

            pickle.dump(d, handle)

            handle.close()



        return d



    except FileNotFoundError:

        print("File could not be found.")

        print("Program terminating.nnPress the enter key to exit.")

I've taken a look at other questions and none seem to touch on the problem I'm facing here. Any help on how to fix it would be greatly appreciated!

Edit:
I'd also like to add that when I run the code and perform

print(initialise_database( #my_directory, #my_file )

It does print out my database, meaning that the update() function is being called, seeing as there is no such database.dat file anywhere

As proof that the file is not appearing, here is a screenshot. This is after full_script (which is where this code is taken from) has been ran.

Evidence there is no file

edited Jan 2 at 20:36

asked Jan 2 at 20:09

Tom_G_99

429

2

Why do you close handle? The withtakes care of that.

– progmatico
Jan 2 at 20:14

he has his return statement inside his with block. meaning it might not close, unless the with did close it.

– GeeTransit
Jan 2 at 20:18

@progmatico haha, thanks for pointing that out. I didn't know that. I'm still quite new to programming... I've taken it out but still no change :( idk why no binary file is being created. I double-checked all my directories and none have the file. Thus, it should create the file!

– Tom_G_99
Jan 2 at 20:20

have you checked in the directory the python file is in?

– GeeTransit
Jan 2 at 20:25

2

@GeeTransit Doesn't directly relate to the question, but with guarantees that the file closes even if there is a return statement.

– Tomothy32
Jan 2 at 20:28

|
show 7 more comments

with open("database.dat", "wb") as handle:

        pickle.dump(d, handle)

        handle.close()

To give an idea of where this fits in, this the function:

import pickle



def initialise_database(directory, file_name):



    try:

        with open("database.dat", "rb") as handle:

            d = pickle.load(handle) # THIS WORKS PERFECTLY

            handle.close()

            return d



    except FileNotFoundError:

        return update_database(directory, file_name)





def update_database(directory, file_name):

    import xlrd, os



    # Change directory

    os.chdir(directory)



    try:

        wb = xlrd.open_workbook(file_name)



        sheet = wb.sheet_by_name("FUNDS")



        rows, cols = sheet.nrows, sheet.ncols



        d = {}

        for i in range(rows - 1):

            if sheet.cell_value(i + 1, 0) != "":

                charity = cap(sheet.cell_value(i + 1, 0))



                area_of_work = 

                org = 

                funding = 0



                for x in range(cols):

                    if sheet.cell_value(0, x) == "Area of work":

                        if sheet.cell_value(i + 1, x) != "":

                            area_of_work.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass



                    elif sheet.cell_value(0, x) == "Organisation funded":

                        if sheet.cell_value(i + 1, x) != "":

                            org.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass





                    elif sheet.cell_value(0, x) == "How much funding provided":

                        funding = (str_to_int(sheet.cell_value(i + 1, x)))



                        if funding is False:

                            print("nCharity " + str(sheet.cell_value(i + 1, 0)) + ", number " + str(i + 2) + " has funding of value: "+ str(funding) + ", which is not compatible. Please find it and fix it in the the excel file. Then try again")

                            return False



                # Adds entry to dictionary

                d[charity] = [area_of_work, org, funding]



            else:

                pass



        # After the loop has finished, it should write the entire dictionary to a newly created file!



        with open("database.dat", "wb") as handle:

            pickle.dump(d, handle)

            handle.close()



        return d



    except FileNotFoundError:

        print("File could not be found.")

        print("Program terminating.nnPress the enter key to exit.")

I've taken a look at other questions and none seem to touch on the problem I'm facing here. Any help on how to fix it would be greatly appreciated!

Edit:
I'd also like to add that when I run the code and perform

print(initialise_database( #my_directory, #my_file )

It does print out my database, meaning that the update() function is being called, seeing as there is no such database.dat file anywhere

As proof that the file is not appearing, here is a screenshot. This is after full_script (which is where this code is taken from) has been ran.

Evidence there is no file

edited Jan 2 at 20:36

asked Jan 2 at 20:09

Tom_G_99

429

with open("database.dat", "wb") as handle:

        pickle.dump(d, handle)

        handle.close()

To give an idea of where this fits in, this the function:

import pickle



def initialise_database(directory, file_name):



    try:

        with open("database.dat", "rb") as handle:

            d = pickle.load(handle) # THIS WORKS PERFECTLY

            handle.close()

            return d



    except FileNotFoundError:

        return update_database(directory, file_name)





def update_database(directory, file_name):

    import xlrd, os



    # Change directory

    os.chdir(directory)



    try:

        wb = xlrd.open_workbook(file_name)



        sheet = wb.sheet_by_name("FUNDS")



        rows, cols = sheet.nrows, sheet.ncols



        d = {}

        for i in range(rows - 1):

            if sheet.cell_value(i + 1, 0) != "":

                charity = cap(sheet.cell_value(i + 1, 0))



                area_of_work = 

                org = 

                funding = 0



                for x in range(cols):

                    if sheet.cell_value(0, x) == "Area of work":

                        if sheet.cell_value(i + 1, x) != "":

                            area_of_work.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass



                    elif sheet.cell_value(0, x) == "Organisation funded":

                        if sheet.cell_value(i + 1, x) != "":

                            org.append(cap(sheet.cell_value(i + 1, x)))

                        else:

                            pass





                    elif sheet.cell_value(0, x) == "How much funding provided":

                        funding = (str_to_int(sheet.cell_value(i + 1, x)))



                        if funding is False:

                            print("nCharity " + str(sheet.cell_value(i + 1, 0)) + ", number " + str(i + 2) + " has funding of value: "+ str(funding) + ", which is not compatible. Please find it and fix it in the the excel file. Then try again")

                            return False



                # Adds entry to dictionary

                d[charity] = [area_of_work, org, funding]



            else:

                pass



        # After the loop has finished, it should write the entire dictionary to a newly created file!



        with open("database.dat", "wb") as handle:

            pickle.dump(d, handle)

            handle.close()



        return d



    except FileNotFoundError:

        print("File could not be found.")

        print("Program terminating.nnPress the enter key to exit.")

I've taken a look at other questions and none seem to touch on the problem I'm facing here. Any help on how to fix it would be greatly appreciated!

Edit:
I'd also like to add that when I run the code and perform

print(initialise_database( #my_directory, #my_file )

It does print out my database, meaning that the update() function is being called, seeing as there is no such database.dat file anywhere

As proof that the file is not appearing, here is a screenshot. This is after full_script (which is where this code is taken from) has been ran.

Evidence there is no file

python io pickle binaryfiles

edited Jan 2 at 20:36

asked Jan 2 at 20:09

Tom_G_99

429

edited Jan 2 at 20:36

asked Jan 2 at 20:09

Tom_G_99

429

edited Jan 2 at 20:36

asked Jan 2 at 20:09

Tom_G_99

429

asked Jan 2 at 20:09

Tom_G_99

429

asked Jan 2 at 20:09

Tom_G_99

429

2

Why do you close handle? The withtakes care of that.

– progmatico
Jan 2 at 20:14

he has his return statement inside his with block. meaning it might not close, unless the with did close it.

– GeeTransit
Jan 2 at 20:18

@progmatico haha, thanks for pointing that out. I didn't know that. I'm still quite new to programming... I've taken it out but still no change :( idk why no binary file is being created. I double-checked all my directories and none have the file. Thus, it should create the file!

– Tom_G_99
Jan 2 at 20:20

have you checked in the directory the python file is in?

– GeeTransit
Jan 2 at 20:25

2

@GeeTransit Doesn't directly relate to the question, but with guarantees that the file closes even if there is a return statement.

– Tomothy32
Jan 2 at 20:28

|
show 7 more comments

2

Why do you close handle? The withtakes care of that.

– progmatico
Jan 2 at 20:14

he has his return statement inside his with block. meaning it might not close, unless the with did close it.

– GeeTransit
Jan 2 at 20:18

@progmatico haha, thanks for pointing that out. I didn't know that. I'm still quite new to programming... I've taken it out but still no change :( idk why no binary file is being created. I double-checked all my directories and none have the file. Thus, it should create the file!

– Tom_G_99
Jan 2 at 20:20

have you checked in the directory the python file is in?

– GeeTransit
Jan 2 at 20:25

2

@GeeTransit Doesn't directly relate to the question, but with guarantees that the file closes even if there is a return statement.

– Tomothy32
Jan 2 at 20:28

Why do you close handle? The withtakes care of that.

– progmatico
Jan 2 at 20:14

he has his return statement inside his with block. meaning it might not close, unless the with did close it.

– GeeTransit
Jan 2 at 20:18

@progmatico haha, thanks for pointing that out. I didn't know that. I'm still quite new to programming... I've taken it out but still no change :( idk why no binary file is being created. I double-checked all my directories and none have the file. Thus, it should create the file!

– Tom_G_99
Jan 2 at 20:20

have you checked in the directory the python file is in?

– GeeTransit
Jan 2 at 20:25

@GeeTransit Doesn't directly relate to the question, but with guarantees that the file closes even if there is a return statement.

– Tomothy32
Jan 2 at 20:28

|
show 7 more comments

1 Answer
1

active

oldest

votes

I would change:

# Change directory

os.chdir(directory)

wb = xlrd.open_workbook(file_name)

to be:

import os.path



wb = xlrd.open_workbook(os.path.join(directory, file_name))

that way the "working directory" won't get changed, and your files will end up where you expect.

to explain a bit: whenever you open() a file (this is what Python ends up calling, as would xlrd or any other native code). paths that don't start with a / (or in Windows) are relative to your "current working directory" (CWD) or just working directory/folder. your CWD will be inherited from the program that started your code (e.g. your shell or IDE) but you can change it programmatically as you were doing using os.chdir.

I generally stay away from changing directory as it tends to cause confusion (as you've experienced) and just use paths that get to the right place. there are lots of better docs on the internet about this. Microsoft has a document called Naming Files, Paths, and Namespaces that has all the gorey details and it's generally easier in Unix/Linux. try searching for "relative vs absolute" path names, and finding out what .. means.

edited Jan 2 at 21:42

answered Jan 2 at 20:58

Sam Mason

3,35811331

I would hug you if I could... Do you mind explaining why the change matter? My knowledge of I/O is quite rudimentary and I would be very grateful if you could give some explanation to the solution

– Tom_G_99
Jan 2 at 21:01

1

@Tom_G_99, it looks like your file was being created in a different folder. Your code has a "working folder", usually the one from where it is launched. With chdir you are changing it. When making I/O with relative paths or just a filename, this happens into the new changed working folder. With the solution given you don't change your working folder, so instead you have to open the files where they are, specifying a path for them.

– progmatico
Jan 2 at 21:29

add a comment |

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54012537%2funable-to-write-to-binary-file-using-pickle-python%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

I would change:

# Change directory

os.chdir(directory)

wb = xlrd.open_workbook(file_name)

to be:

import os.path



wb = xlrd.open_workbook(os.path.join(directory, file_name))

that way the "working directory" won't get changed, and your files will end up where you expect.

edited Jan 2 at 21:42

answered Jan 2 at 20:58

Sam Mason

3,35811331

I would hug you if I could... Do you mind explaining why the change matter? My knowledge of I/O is quite rudimentary and I would be very grateful if you could give some explanation to the solution

– Tom_G_99
Jan 2 at 21:01

1

@Tom_G_99, it looks like your file was being created in a different folder. Your code has a "working folder", usually the one from where it is launched. With chdir you are changing it. When making I/O with relative paths or just a filename, this happens into the new changed working folder. With the solution given you don't change your working folder, so instead you have to open the files where they are, specifying a path for them.

– progmatico
Jan 2 at 21:29

add a comment |

I would change:

# Change directory

os.chdir(directory)

wb = xlrd.open_workbook(file_name)

to be:

import os.path



wb = xlrd.open_workbook(os.path.join(directory, file_name))

that way the "working directory" won't get changed, and your files will end up where you expect.

edited Jan 2 at 21:42

answered Jan 2 at 20:58

Sam Mason

3,35811331

I would hug you if I could... Do you mind explaining why the change matter? My knowledge of I/O is quite rudimentary and I would be very grateful if you could give some explanation to the solution

– Tom_G_99
Jan 2 at 21:01

1

@Tom_G_99, it looks like your file was being created in a different folder. Your code has a "working folder", usually the one from where it is launched. With chdir you are changing it. When making I/O with relative paths or just a filename, this happens into the new changed working folder. With the solution given you don't change your working folder, so instead you have to open the files where they are, specifying a path for them.

– progmatico
Jan 2 at 21:29

add a comment |

I would change:

# Change directory

os.chdir(directory)

wb = xlrd.open_workbook(file_name)

to be:

import os.path



wb = xlrd.open_workbook(os.path.join(directory, file_name))

that way the "working directory" won't get changed, and your files will end up where you expect.

edited Jan 2 at 21:42

answered Jan 2 at 20:58

Sam Mason

3,35811331

I would change:

# Change directory

os.chdir(directory)

wb = xlrd.open_workbook(file_name)

to be:

import os.path



wb = xlrd.open_workbook(os.path.join(directory, file_name))

that way the "working directory" won't get changed, and your files will end up where you expect.

edited Jan 2 at 21:42

answered Jan 2 at 20:58

Sam Mason

3,35811331

edited Jan 2 at 21:42

answered Jan 2 at 20:58

Sam Mason

3,35811331

answered Jan 2 at 20:58

Sam Mason

3,35811331

answered Jan 2 at 20:58

Sam Mason

3,35811331

I would hug you if I could... Do you mind explaining why the change matter? My knowledge of I/O is quite rudimentary and I would be very grateful if you could give some explanation to the solution

– Tom_G_99
Jan 2 at 21:01

1

@Tom_G_99, it looks like your file was being created in a different folder. Your code has a "working folder", usually the one from where it is launched. With chdir you are changing it. When making I/O with relative paths or just a filename, this happens into the new changed working folder. With the solution given you don't change your working folder, so instead you have to open the files where they are, specifying a path for them.

– progmatico
Jan 2 at 21:29

add a comment |

I would hug you if I could... Do you mind explaining why the change matter? My knowledge of I/O is quite rudimentary and I would be very grateful if you could give some explanation to the solution

– Tom_G_99
Jan 2 at 21:01

1

@Tom_G_99, it looks like your file was being created in a different folder. Your code has a "working folder", usually the one from where it is launched. With chdir you are changing it. When making I/O with relative paths or just a filename, this happens into the new changed working folder. With the solution given you don't change your working folder, so instead you have to open the files where they are, specifying a path for them.

– progmatico
Jan 2 at 21:29

I would hug you if I could... Do you mind explaining why the change matter? My knowledge of I/O is quite rudimentary and I would be very grateful if you could give some explanation to the solution

– Tom_G_99
Jan 2 at 21:01

@Tom_G_99, it looks like your file was being created in a different folder. Your code has a "working folder", usually the one from where it is launched. With chdir you are changing it. When making I/O with relative paths or just a filename, this happens into the new changed working folder. With the solution given you don't change your working folder, so instead you have to open the files where they are, specifying a path for them.

– progmatico
Jan 2 at 21:29

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu