Mixing
Scala: Json sorting operations on a json file which has different datatypes but same schema
I have a json file which has two types of data for store and online.But datatypes of few columns are different ,but column names and schema are same.
I would like to do some operations on json like sorting,re-arranging using spray joson library in Scala.But I am not able to apply the case class on josn becuase datatype of "storeId" and "amount" are different.
Is there any way we can re-write case class to handle bothe integer and string ?
Example : I will have them in a file, read it line by line and assign it to string
Given inputJosn String :
{
"purchaseType": "1",
"purchaseChannel": "store",
"storeId": 6167,
"paymentType": [{
"type": "CASH",
"category": "Cash",
"amount": 3.91
}]
},
{
"purchaseType": "2",
"purchaseChannel": "online",
"storeId": "6168",
"paymentType": [{
"type": "Card",
"category": "Card",
"amount": "5.04"
}]
}
Code :
import spray.json._
import DefaultJsonProtocol._
case class payType(`type`:String,category:String,amount:String)
case class Transactions(purchaseType:String,purchaseChannel:String,storeId:String,paymentType: payType )
object MyJsonpayType extends DefaultJsonProtocol {
implicit val payTypeFormat: JsonFormat[payType] = jsonFormat3(payType)
}
import MyJsonpayType._
object MyJsonTransactions extends DefaultJsonProtocol {
implicit val TransactionsFormat: JsonFormat[Transactions] = jsonFormat4(Transactions)
}
import MyJsonTransactions._
inputJosn.parseJson.convertTo[Transactions]
It throws errror saying expecing JsString but got 6167.
I know why is it throwing error, but is these any way to re-writw case class in bettwer way to handle both integer and string.
json scala spray-json
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
add a comment |
I have a json file which has two types of data for store and online.But datatypes of few columns are different ,but column names and schema are same.
I would like to do some operations on json like sorting,re-arranging using spray joson library in Scala.But I am not able to apply the case class on josn becuase datatype of "storeId" and "amount" are different.
Is there any way we can re-write case class to handle bothe integer and string ?
Example : I will have them in a file, read it line by line and assign it to string
Given inputJosn String :
{
"purchaseType": "1",
"purchaseChannel": "store",
"storeId": 6167,
"paymentType": [{
"type": "CASH",
"category": "Cash",
"amount": 3.91
}]
},
{
"purchaseType": "2",
"purchaseChannel": "online",
"storeId": "6168",
"paymentType": [{
"type": "Card",
"category": "Card",
"amount": "5.04"
}]
}
Code :
import spray.json._
import DefaultJsonProtocol._
case class payType(`type`:String,category:String,amount:String)
case class Transactions(purchaseType:String,purchaseChannel:String,storeId:String,paymentType: payType )
object MyJsonpayType extends DefaultJsonProtocol {
implicit val payTypeFormat: JsonFormat[payType] = jsonFormat3(payType)
}
import MyJsonpayType._
object MyJsonTransactions extends DefaultJsonProtocol {
implicit val TransactionsFormat: JsonFormat[Transactions] = jsonFormat4(Transactions)
}
import MyJsonTransactions._
inputJosn.parseJson.convertTo[Transactions]
It throws errror saying expecing JsString but got 6167.
I know why is it throwing error, but is these any way to re-writw case class in bettwer way to handle both integer and string.
json scala spray-json
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
type T >: String with Int why can't you use this for storeId?
– Raman Mishra
Nov 20 '18 at 6:39
add a comment |
I have a json file which has two types of data for store and online.But datatypes of few columns are different ,but column names and schema are same.
I would like to do some operations on json like sorting,re-arranging using spray joson library in Scala.But I am not able to apply the case class on josn becuase datatype of "storeId" and "amount" are different.
Is there any way we can re-write case class to handle bothe integer and string ?
Example : I will have them in a file, read it line by line and assign it to string
Given inputJosn String :
{
"purchaseType": "1",
"purchaseChannel": "store",
"storeId": 6167,
"paymentType": [{
"type": "CASH",
"category": "Cash",
"amount": 3.91
}]
},
{
"purchaseType": "2",
"purchaseChannel": "online",
"storeId": "6168",
"paymentType": [{
"type": "Card",
"category": "Card",
"amount": "5.04"
}]
}
Code :
import spray.json._
import DefaultJsonProtocol._
case class payType(`type`:String,category:String,amount:String)
case class Transactions(purchaseType:String,purchaseChannel:String,storeId:String,paymentType: payType )
object MyJsonpayType extends DefaultJsonProtocol {
implicit val payTypeFormat: JsonFormat[payType] = jsonFormat3(payType)
}
import MyJsonpayType._
object MyJsonTransactions extends DefaultJsonProtocol {
implicit val TransactionsFormat: JsonFormat[Transactions] = jsonFormat4(Transactions)
}
import MyJsonTransactions._
inputJosn.parseJson.convertTo[Transactions]
It throws errror saying expecing JsString but got 6167.
I know why is it throwing error, but is these any way to re-writw case class in bettwer way to handle both integer and string.
json scala spray-json
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
I have a json file which has two types of data for store and online.But datatypes of few columns are different ,but column names and schema are same.
I would like to do some operations on json like sorting,re-arranging using spray joson library in Scala.But I am not able to apply the case class on josn becuase datatype of "storeId" and "amount" are different.
Is there any way we can re-write case class to handle bothe integer and string ?
Example : I will have them in a file, read it line by line and assign it to string
Given inputJosn String :
{
"purchaseType": "1",
"purchaseChannel": "store",
"storeId": 6167,
"paymentType": [{
"type": "CASH",
"category": "Cash",
"amount": 3.91
}]
},
{
"purchaseType": "2",
"purchaseChannel": "online",
"storeId": "6168",
"paymentType": [{
"type": "Card",
"category": "Card",
"amount": "5.04"
}]
}
Code :
import spray.json._
import DefaultJsonProtocol._
case class payType(`type`:String,category:String,amount:String)
case class Transactions(purchaseType:String,purchaseChannel:String,storeId:String,paymentType: payType )
object MyJsonpayType extends DefaultJsonProtocol {
implicit val payTypeFormat: JsonFormat[payType] = jsonFormat3(payType)
}
import MyJsonpayType._
object MyJsonTransactions extends DefaultJsonProtocol {
implicit val TransactionsFormat: JsonFormat[Transactions] = jsonFormat4(Transactions)
}
import MyJsonTransactions._
inputJosn.parseJson.convertTo[Transactions]
It throws errror saying expecing JsString but got 6167.
I know why is it throwing error, but is these any way to re-writw case class in bettwer way to handle both integer and string.
json scala spray-json
json scala spray-json
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
asked Nov 20 '18 at 6:22
Rohan NayakRohan Nayak
11611
11611
type T >: String with Int why can't you use this for storeId?
– Raman Mishra
Nov 20 '18 at 6:39
add a comment |
type T >: String with Int why can't you use this for storeId?
– Raman Mishra
Nov 20 '18 at 6:39
type T >: String with Int why can't you use this for storeId?
– Raman Mishra
Nov 20 '18 at 6:39
type T >: String with Int why can't you use this for storeId?
– Raman Mishra
Nov 20 '18 at 6:39
add a comment |
2 Answers
2
active
oldest
votes
You can get away with using Either, like
case class payType(`type`:String,category:String,amount:Either[Double,String])
But then you are simply kicking the can down the road. If you have to do any operation on the amount field like- sort, aggregation you would need to normalise the data.
If that's case I would better write custom de-serialization for the object with having a single type.
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
add a comment |
In these situations I like to use the Jackson library, but in this situation you could just call the storeId:string. After that you can parse it to long by using a companion object.
Try this:
case class Transactions (
purchaseType:String,
purchaseChannel:String,
storeId:String,
paymentType: payType ) {
def storeIdToLong(t: Transaction) = TransactionsAsLong(
purchaseType = t.purchaseType,
purchaseChannel = t.purchaseChannel,
storeId = t.storeId.toLong,
paymentType = t.paymentType
)
}
case class TransactionsAsLong (
purchaseType:String,
purchaseChannel:String,
storeId:Long,
paymentType: payType )
object Transactions {
implicit def transactions_to_long(t: Transactions): TransactionsAsLong = t.storeIdToLong()
}
Now you can write val test: Transaction = TransactionsAsLong(...) if you need that.
I would suggest to write a custom deserializer instead :-)
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53387339%2fscala-json-sorting-operations-on-a-json-file-which-has-different-datatypes-but%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can get away with using Either, like
case class payType(`type`:String,category:String,amount:Either[Double,String])
But then you are simply kicking the can down the road. If you have to do any operation on the amount field like- sort, aggregation you would need to normalise the data.
If that's case I would better write custom de-serialization for the object with having a single type.
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
add a comment |
You can get away with using Either, like
case class payType(`type`:String,category:String,amount:Either[Double,String])
But then you are simply kicking the can down the road. If you have to do any operation on the amount field like- sort, aggregation you would need to normalise the data.
If that's case I would better write custom de-serialization for the object with having a single type.
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
add a comment |
You can get away with using Either, like
case class payType(`type`:String,category:String,amount:Either[Double,String])
But then you are simply kicking the can down the road. If you have to do any operation on the amount field like- sort, aggregation you would need to normalise the data.
If that's case I would better write custom de-serialization for the object with having a single type.
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
You can get away with using Either, like
case class payType(`type`:String,category:String,amount:Either[Double,String])
But then you are simply kicking the can down the road. If you have to do any operation on the amount field like- sort, aggregation you would need to normalise the data.
If that's case I would better write custom de-serialization for the object with having a single type.
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
edited Nov 20 '18 at 15:13
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
answered Nov 20 '18 at 6:56
BiswanathBiswanath
5,024103856
5,024103856
add a comment |
add a comment |
In these situations I like to use the Jackson library, but in this situation you could just call the storeId:string. After that you can parse it to long by using a companion object.
Try this:
case class Transactions (
purchaseType:String,
purchaseChannel:String,
storeId:String,
paymentType: payType ) {
def storeIdToLong(t: Transaction) = TransactionsAsLong(
purchaseType = t.purchaseType,
purchaseChannel = t.purchaseChannel,
storeId = t.storeId.toLong,
paymentType = t.paymentType
)
}
case class TransactionsAsLong (
purchaseType:String,
purchaseChannel:String,
storeId:Long,
paymentType: payType )
object Transactions {
implicit def transactions_to_long(t: Transactions): TransactionsAsLong = t.storeIdToLong()
}
Now you can write val test: Transaction = TransactionsAsLong(...) if you need that.
I would suggest to write a custom deserializer instead :-)
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
add a comment |
In these situations I like to use the Jackson library, but in this situation you could just call the storeId:string. After that you can parse it to long by using a companion object.
Try this:
case class Transactions (
purchaseType:String,
purchaseChannel:String,
storeId:String,
paymentType: payType ) {
def storeIdToLong(t: Transaction) = TransactionsAsLong(
purchaseType = t.purchaseType,
purchaseChannel = t.purchaseChannel,
storeId = t.storeId.toLong,
paymentType = t.paymentType
)
}
case class TransactionsAsLong (
purchaseType:String,
purchaseChannel:String,
storeId:Long,
paymentType: payType )
object Transactions {
implicit def transactions_to_long(t: Transactions): TransactionsAsLong = t.storeIdToLong()
}
Now you can write val test: Transaction = TransactionsAsLong(...) if you need that.
I would suggest to write a custom deserializer instead :-)
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
add a comment |
In these situations I like to use the Jackson library, but in this situation you could just call the storeId:string. After that you can parse it to long by using a companion object.
Try this:
case class Transactions (
purchaseType:String,
purchaseChannel:String,
storeId:String,
paymentType: payType ) {
def storeIdToLong(t: Transaction) = TransactionsAsLong(
purchaseType = t.purchaseType,
purchaseChannel = t.purchaseChannel,
storeId = t.storeId.toLong,
paymentType = t.paymentType
)
}
case class TransactionsAsLong (
purchaseType:String,
purchaseChannel:String,
storeId:Long,
paymentType: payType )
object Transactions {
implicit def transactions_to_long(t: Transactions): TransactionsAsLong = t.storeIdToLong()
}
Now you can write val test: Transaction = TransactionsAsLong(...) if you need that.
I would suggest to write a custom deserializer instead :-)
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
In these situations I like to use the Jackson library, but in this situation you could just call the storeId:string. After that you can parse it to long by using a companion object.
Try this:
case class Transactions (
purchaseType:String,
purchaseChannel:String,
storeId:String,
paymentType: payType ) {
def storeIdToLong(t: Transaction) = TransactionsAsLong(
purchaseType = t.purchaseType,
purchaseChannel = t.purchaseChannel,
storeId = t.storeId.toLong,
paymentType = t.paymentType
)
}
case class TransactionsAsLong (
purchaseType:String,
purchaseChannel:String,
storeId:Long,
paymentType: payType )
object Transactions {
implicit def transactions_to_long(t: Transactions): TransactionsAsLong = t.storeIdToLong()
}
Now you can write val test: Transaction = TransactionsAsLong(...) if you need that.
I would suggest to write a custom deserializer instead :-)
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
answered Nov 20 '18 at 6:58
Mr.TurtleMr.Turtle
9682925
9682925
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53387339%2fscala-json-sorting-operations-on-a-json-file-which-has-different-datatypes-but%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
type T >: String with Int why can't you use this for storeId?
– Raman Mishra
Nov 20 '18 at 6:39