Azure Web Jobs suddenly stopped working properly

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}

I wrote a program named "blobsUploader" that uploads a csv file to a blob container every night at 11pm.

Whenever a new csv file arrives to the blobs container, a new message in a queue named "blobsAdressQueue" appears with the address of the new blob (csv file).

This evokes the Web Job which reads the csv file and stores all it's data in Azure Table named "myDataTable".

The whole process worked great but suddenly from the past month or two, every night when a new csv is uploaded, there is an error with the Web Job process and the message from "blobsAddressQueue" moves to "blobsAddressQueue-poison" which means a message that has exceeded the maximum number of delivery attempts to the application.

I uploaded now a csv from June 2018, that worked for sure.
However, now the message with the address the this blob is in "blobsAddressQueue-poison".

When I check the logs I can see 5 failing calls:
enter image description here

When I go into one of the attempts and open "Toggle Output", that is what I get:
enter image description here
Which is very weird because this file was read a in June 2018!!! with no any problems! I did not change anything in my code or the csv file since then.

If more information is needed in order to answer the problem, pls let me know.

edited Jan 3 at 12:59

asked Jan 3 at 12:07

I.zv

13510

1

Is the CsvHelper an external reference, like a NuGet package? And if it is, have you updated this package? Because apparently something changed, otherwise you would be having this issue now.

– rickvdbosch
Jan 3 at 12:12

@rickvdbosch yes it is. I am afraid to update this package so I won't make more mess. Also, if everything worked until now with the old package and nothing else was updated, how could it help to update csvHelper ?

– I.zv
Jan 3 at 12:18

On top of that, I just tried another csv from 2017 (their structure is exactly the same) and it worked well.. @rickvdbosch

– I.zv
Jan 3 at 12:21

1

Agree with @rickvdbosch, its your assemblies that changed and likely has nothing to do with the webjobs not working.

– ElvisLives
Jan 3 at 16:03

1

@I.zv I posted an answer weeks ago. Do you have a chance to take a look? If it's helpful, coudl you mark it as an answer so others would see it easily?

– axfd
Feb 18 at 3:13

|
show 3 more comments

I wrote a program named "blobsUploader" that uploads a csv file to a blob container every night at 11pm.

Whenever a new csv file arrives to the blobs container, a new message in a queue named "blobsAdressQueue" appears with the address of the new blob (csv file).

This evokes the Web Job which reads the csv file and stores all it's data in Azure Table named "myDataTable".

I uploaded now a csv from June 2018, that worked for sure.
However, now the message with the address the this blob is in "blobsAddressQueue-poison".

When I check the logs I can see 5 failing calls:
enter image description here

If more information is needed in order to answer the problem, pls let me know.

edited Jan 3 at 12:59

asked Jan 3 at 12:07

I.zv

13510

1

Is the CsvHelper an external reference, like a NuGet package? And if it is, have you updated this package? Because apparently something changed, otherwise you would be having this issue now.

– rickvdbosch
Jan 3 at 12:12

@rickvdbosch yes it is. I am afraid to update this package so I won't make more mess. Also, if everything worked until now with the old package and nothing else was updated, how could it help to update csvHelper ?

– I.zv
Jan 3 at 12:18

On top of that, I just tried another csv from 2017 (their structure is exactly the same) and it worked well.. @rickvdbosch

– I.zv
Jan 3 at 12:21

1

Agree with @rickvdbosch, its your assemblies that changed and likely has nothing to do with the webjobs not working.

– ElvisLives
Jan 3 at 16:03

1

@I.zv I posted an answer weeks ago. Do you have a chance to take a look? If it's helpful, coudl you mark it as an answer so others would see it easily?

– axfd
Feb 18 at 3:13

|
show 3 more comments

I wrote a program named "blobsUploader" that uploads a csv file to a blob container every night at 11pm.

Whenever a new csv file arrives to the blobs container, a new message in a queue named "blobsAdressQueue" appears with the address of the new blob (csv file).

This evokes the Web Job which reads the csv file and stores all it's data in Azure Table named "myDataTable".

I uploaded now a csv from June 2018, that worked for sure.
However, now the message with the address the this blob is in "blobsAddressQueue-poison".

When I check the logs I can see 5 failing calls:
enter image description here

If more information is needed in order to answer the problem, pls let me know.

edited Jan 3 at 12:59

asked Jan 3 at 12:07

I.zv

13510

I wrote a program named "blobsUploader" that uploads a csv file to a blob container every night at 11pm.

Whenever a new csv file arrives to the blobs container, a new message in a queue named "blobsAdressQueue" appears with the address of the new blob (csv file).

This evokes the Web Job which reads the csv file and stores all it's data in Azure Table named "myDataTable".

I uploaded now a csv from June 2018, that worked for sure.
However, now the message with the address the this blob is in "blobsAddressQueue-poison".

When I check the logs I can see 5 failing calls:
enter image description here

If more information is needed in order to answer the problem, pls let me know.

azure csv web-applications azure-webjobs

edited Jan 3 at 12:59

asked Jan 3 at 12:07

I.zv

13510

edited Jan 3 at 12:59

asked Jan 3 at 12:07

I.zv

13510

edited Jan 3 at 12:59

asked Jan 3 at 12:07

I.zv

13510

asked Jan 3 at 12:07

I.zv

13510

asked Jan 3 at 12:07

I.zv

13510

1

Is the CsvHelper an external reference, like a NuGet package? And if it is, have you updated this package? Because apparently something changed, otherwise you would be having this issue now.

– rickvdbosch
Jan 3 at 12:12

@rickvdbosch yes it is. I am afraid to update this package so I won't make more mess. Also, if everything worked until now with the old package and nothing else was updated, how could it help to update csvHelper ?

– I.zv
Jan 3 at 12:18

On top of that, I just tried another csv from 2017 (their structure is exactly the same) and it worked well.. @rickvdbosch

– I.zv
Jan 3 at 12:21

1

Agree with @rickvdbosch, its your assemblies that changed and likely has nothing to do with the webjobs not working.

– ElvisLives
Jan 3 at 16:03

1

@I.zv I posted an answer weeks ago. Do you have a chance to take a look? If it's helpful, coudl you mark it as an answer so others would see it easily?

– axfd
Feb 18 at 3:13

|
show 3 more comments

1

Is the CsvHelper an external reference, like a NuGet package? And if it is, have you updated this package? Because apparently something changed, otherwise you would be having this issue now.

– rickvdbosch
Jan 3 at 12:12

@rickvdbosch yes it is. I am afraid to update this package so I won't make more mess. Also, if everything worked until now with the old package and nothing else was updated, how could it help to update csvHelper ?

– I.zv
Jan 3 at 12:18

On top of that, I just tried another csv from 2017 (their structure is exactly the same) and it worked well.. @rickvdbosch

– I.zv
Jan 3 at 12:21

1

Agree with @rickvdbosch, its your assemblies that changed and likely has nothing to do with the webjobs not working.

– ElvisLives
Jan 3 at 16:03

1

@I.zv I posted an answer weeks ago. Do you have a chance to take a look? If it's helpful, coudl you mark it as an answer so others would see it easily?

– axfd
Feb 18 at 3:13

Is the CsvHelper an external reference, like a NuGet package? And if it is, have you updated this package? Because apparently something changed, otherwise you would be having this issue now.

– rickvdbosch
Jan 3 at 12:12

@rickvdbosch yes it is. I am afraid to update this package so I won't make more mess. Also, if everything worked until now with the old package and nothing else was updated, how could it help to update csvHelper ?

– I.zv
Jan 3 at 12:18

On top of that, I just tried another csv from 2017 (their structure is exactly the same) and it worked well.. @rickvdbosch

– I.zv
Jan 3 at 12:21

Agree with @rickvdbosch, its your assemblies that changed and likely has nothing to do with the webjobs not working.

– ElvisLives
Jan 3 at 16:03

@I.zv I posted an answer weeks ago. Do you have a chance to take a look? If it's helpful, coudl you mark it as an answer so others would see it easily?

– axfd
Feb 18 at 3:13

|
show 3 more comments

1 Answer
1

active

oldest

votes

This issue is not related to webjob, but CsvHelper library you reference to. I've checked the source code and found a field will be considered as bad data when it contains quote and the field is not quoted (escaped).

Source code:

/// <summary>

/// Gets or sets the function that is called when bad field data is found. A field

/// has bad data if it contains a quote and the field is not quoted (escaped).

/// You can supply your own function to do other things like logging the issue

/// instead of throwing an exception.

/// Arguments: context

/// </summary>

Action<ReadingContext> BadDataFound { get; set; }

The solution is

Modify the problematic fields in csv file

Ingnore the bad data by setting BadDataFound to null:

csv.Configuration.BadDataFound = null;

Sample code：

    static void Main(string args)

    {

        using (var reader = new StreamReader(@"C:UserstomlDesktoptest.csv"))

        using (var csv = new CsvReader(reader))

        {

            csv.Configuration.BadDataFound = null;

            var records = csv.GetRecords<Foo>();



            foreach(var item in records)

            {

                Console.WriteLine(item.Name);

            }

        }



        Console.ReadKey();

    }

}



public class Foo

{

    public int Id { get; set; }

    public string Name { get; set; }

}

Invalid CSV sample:

    Id,Name

    1,one"

    2,two

Valid CSV sample:

    Id,Name

    1,"one""

    2,two

answered Jan 15 at 10:12

axfd

1575

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54021998%2fazure-web-jobs-suddenly-stopped-working-properly%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

Source code:

/// <summary>

/// Gets or sets the function that is called when bad field data is found. A field

/// has bad data if it contains a quote and the field is not quoted (escaped).

/// You can supply your own function to do other things like logging the issue

/// instead of throwing an exception.

/// Arguments: context

/// </summary>

Action<ReadingContext> BadDataFound { get; set; }

The solution is

Modify the problematic fields in csv file

Ingnore the bad data by setting BadDataFound to null:

csv.Configuration.BadDataFound = null;

Sample code：

    static void Main(string args)

    {

        using (var reader = new StreamReader(@"C:UserstomlDesktoptest.csv"))

        using (var csv = new CsvReader(reader))

        {

            csv.Configuration.BadDataFound = null;

            var records = csv.GetRecords<Foo>();



            foreach(var item in records)

            {

                Console.WriteLine(item.Name);

            }

        }



        Console.ReadKey();

    }

}



public class Foo

{

    public int Id { get; set; }

    public string Name { get; set; }

}

Invalid CSV sample:

    Id,Name

    1,one"

    2,two

Valid CSV sample:

    Id,Name

    1,"one""

    2,two

answered Jan 15 at 10:12

axfd

1575

add a comment |

Source code:

/// <summary>

/// Gets or sets the function that is called when bad field data is found. A field

/// has bad data if it contains a quote and the field is not quoted (escaped).

/// You can supply your own function to do other things like logging the issue

/// instead of throwing an exception.

/// Arguments: context

/// </summary>

Action<ReadingContext> BadDataFound { get; set; }

The solution is

Modify the problematic fields in csv file

Ingnore the bad data by setting BadDataFound to null:

csv.Configuration.BadDataFound = null;

Sample code：

    static void Main(string args)

    {

        using (var reader = new StreamReader(@"C:UserstomlDesktoptest.csv"))

        using (var csv = new CsvReader(reader))

        {

            csv.Configuration.BadDataFound = null;

            var records = csv.GetRecords<Foo>();



            foreach(var item in records)

            {

                Console.WriteLine(item.Name);

            }

        }



        Console.ReadKey();

    }

}



public class Foo

{

    public int Id { get; set; }

    public string Name { get; set; }

}

Invalid CSV sample:

    Id,Name

    1,one"

    2,two

Valid CSV sample:

    Id,Name

    1,"one""

    2,two

answered Jan 15 at 10:12

axfd

1575

add a comment |

Source code:

/// <summary>

/// Gets or sets the function that is called when bad field data is found. A field

/// has bad data if it contains a quote and the field is not quoted (escaped).

/// You can supply your own function to do other things like logging the issue

/// instead of throwing an exception.

/// Arguments: context

/// </summary>

Action<ReadingContext> BadDataFound { get; set; }

The solution is

Modify the problematic fields in csv file

Ingnore the bad data by setting BadDataFound to null:

csv.Configuration.BadDataFound = null;

Sample code：

    static void Main(string args)

    {

        using (var reader = new StreamReader(@"C:UserstomlDesktoptest.csv"))

        using (var csv = new CsvReader(reader))

        {

            csv.Configuration.BadDataFound = null;

            var records = csv.GetRecords<Foo>();



            foreach(var item in records)

            {

                Console.WriteLine(item.Name);

            }

        }



        Console.ReadKey();

    }

}



public class Foo

{

    public int Id { get; set; }

    public string Name { get; set; }

}

Invalid CSV sample:

    Id,Name

    1,one"

    2,two

Valid CSV sample:

    Id,Name

    1,"one""

    2,two

answered Jan 15 at 10:12

axfd

1575

Source code:

/// <summary>

/// Gets or sets the function that is called when bad field data is found. A field

/// has bad data if it contains a quote and the field is not quoted (escaped).

/// You can supply your own function to do other things like logging the issue

/// instead of throwing an exception.

/// Arguments: context

/// </summary>

Action<ReadingContext> BadDataFound { get; set; }

The solution is

Modify the problematic fields in csv file

Ingnore the bad data by setting BadDataFound to null:

csv.Configuration.BadDataFound = null;

Sample code：

    static void Main(string args)

    {

        using (var reader = new StreamReader(@"C:UserstomlDesktoptest.csv"))

        using (var csv = new CsvReader(reader))

        {

            csv.Configuration.BadDataFound = null;

            var records = csv.GetRecords<Foo>();



            foreach(var item in records)

            {

                Console.WriteLine(item.Name);

            }

        }



        Console.ReadKey();

    }

}



public class Foo

{

    public int Id { get; set; }

    public string Name { get; set; }

}

Invalid CSV sample:

    Id,Name

    1,one"

    2,two

Valid CSV sample:

    Id,Name

    1,"one""

    2,two

answered Jan 15 at 10:12

axfd

1575

answered Jan 15 at 10:12

axfd

1575

answered Jan 15 at 10:12

axfd

1575

answered Jan 15 at 10:12

axfd

1575

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu