Styjun

Why has "pence" been used in this sentence, not "pences"?

Divine apple island

Proving a function is onto where f(x)=|x|.

Can I use my Chinese passport to enter China after I acquired another citizenship?

Should I install hardwood flooring or cabinets first?

When quoting, must I also copy hyphens used to divide words that continue on the next line?

Transformation of random variables and joint distributions

MAXDOP Settings for SQL Server 2014

Why did the EU agree to delay the Brexit deadline?

Can someone explain how this makes sense electrically?

Why does the integral domain "being trapped between a finite field extension" implies that it is a field?

My friend sent me a screenshot of a transaction hash, but when I search for it I find divergent data. What happened?

Journal losing indexing services

Varistor? Purpose and principle

What linear sensor for a keyboard?

Why in book's example is used 言葉(ことば) instead of 言語(げんご)?

Indicating multiple different modes of speech (fantasy language or telepathy)

Difference between -| and |- in TikZ

List of people who lose a child in תנ"ך

Have I saved too much for retirement so far?

Why do IPv6 unique local addresses have to have a /48 prefix?

How should I respond when I lied about my education and the company finds out through background check?

A Permanent Norse Presence in America

Is it possible to have a strip of cold climate in the middle of a planet?

Survival AFT Analysis with Scala

Scala vs. Groovy vs. ClojureIs the Scala 2.8 collections library a case of “the longest suicide note in history”?Difference between object and class in ScalaWhat are all the uses of an underscore in Scala?Invalid survival times for distribution, Survival Analysis; package survivalRepresenting Parametric Survival Model in 'Counting Process' form in JAGScalculate p-value for AFT survival model in Sparkforecast time to event survival analysisCreat survival object with both right-cens and left-truncated + right-cens observationsDifferent results when presenting “From-To” data in survival analysis

2

I am trying to implement the survival analysis model as documented here: Scala-Docs#Survival-Regression but I cannot make heads or tails of how you are supposed to do the actual implementation.

I am trying to model the "survivability" of a customer for a business. Survivability of a customer is a label given to customers based on if a purchase was made in the last month. If a customer fails to make a purchase, they are considered dead/censured. The two factors I am taking into account are "number of times advertised to" and "amount of time spent on business website". Data is collected about the customer on a monthly basis.

Here is what my data looks like for two customers (CustA and CustB) over three monthly time periods:

val seqCust = Seq(
//Customer,Period,Censor,# of Ads,Amount of Time on Site
("CustA",1,0,4,2400),
("CustA",2,0,6,1800),
("CustA",3,1,2,600),
("CustB",1,0,2,2800),
("CustB",2,0,4,2100),
("CustB",3,0,3,1200)
)

I then want to transform it into something like this as the docs specify:

val dfCust = seqCust.map(cr=>(cr._2,cr._3,Vectors.dense(cr._4,cr._5)).toDF("label", "censor", "features")

So that my data now looks like this:

[1,0,[4,2400]],
[2,0,[6,1800]],
[3,1,[2,600]],
[1,0,[2,2800]],
[2,0,[4,2100]],
[3,0,[3,1200]]

And then do the following:

val quantileProbabilities = Array(0.3, 0.6)
val aft = new AFTSurvivalRegression()
 .setQuantileProbabilities(quantileProbabilities)
 .setQuantilesCol("quantiles")

val model = aft.fit(dfCust)

// Print the coefficients, intercept and scale parameter for AFT survival regression
println(s"Coefficients: $model.coefficients")
println(s"Intercept: $model.intercept")
println(s"Scale: $model.scale")
model.transform(dfCust).show(false)

But I do not understand:

1. Is this the correct way to model the data as per Scala's documentation?


2. How come I am not taking the customer ID into account anywhere?

scala apache-spark survival-analysis survival

share|improve this question

edited Mar 21 at 20:37

EliSquared

asked Mar 21 at 3:05

EliSquaredEliSquared

346516

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I'm not sure about your first question, it isn't clear for me. As per your second question, the default label, censor and features columns are respectively "label", "censor" and "features". That's why you didn't need to precise that explicitly.
  
  – eliasah
  Mar 21 at 9:30
  
  

  
  
  
  

add a comment | 

2

I am trying to implement the survival analysis model as documented here: Scala-Docs#Survival-Regression but I cannot make heads or tails of how you are supposed to do the actual implementation.

I am trying to model the "survivability" of a customer for a business. Survivability of a customer is a label given to customers based on if a purchase was made in the last month. If a customer fails to make a purchase, they are considered dead/censured. The two factors I am taking into account are "number of times advertised to" and "amount of time spent on business website". Data is collected about the customer on a monthly basis.

Here is what my data looks like for two customers (CustA and CustB) over three monthly time periods:

val seqCust = Seq(
//Customer,Period,Censor,# of Ads,Amount of Time on Site
("CustA",1,0,4,2400),
("CustA",2,0,6,1800),
("CustA",3,1,2,600),
("CustB",1,0,2,2800),
("CustB",2,0,4,2100),
("CustB",3,0,3,1200)
)

I then want to transform it into something like this as the docs specify:

val dfCust = seqCust.map(cr=>(cr._2,cr._3,Vectors.dense(cr._4,cr._5)).toDF("label", "censor", "features")

So that my data now looks like this:

[1,0,[4,2400]],
[2,0,[6,1800]],
[3,1,[2,600]],
[1,0,[2,2800]],
[2,0,[4,2100]],
[3,0,[3,1200]]

And then do the following:

val quantileProbabilities = Array(0.3, 0.6)
val aft = new AFTSurvivalRegression()
 .setQuantileProbabilities(quantileProbabilities)
 .setQuantilesCol("quantiles")

val model = aft.fit(dfCust)

// Print the coefficients, intercept and scale parameter for AFT survival regression
println(s"Coefficients: $model.coefficients")
println(s"Intercept: $model.intercept")
println(s"Scale: $model.scale")
model.transform(dfCust).show(false)

But I do not understand:

1. Is this the correct way to model the data as per Scala's documentation?


2. How come I am not taking the customer ID into account anywhere?

scala apache-spark survival-analysis survival

share|improve this question

edited Mar 21 at 20:37

EliSquared

asked Mar 21 at 3:05

EliSquaredEliSquared

346516

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I'm not sure about your first question, it isn't clear for me. As per your second question, the default label, censor and features columns are respectively "label", "censor" and "features". That's why you didn't need to precise that explicitly.
  
  – eliasah
  Mar 21 at 9:30
  
  

  
  
  
  

add a comment | 

I am trying to implement the survival analysis model as documented here: Scala-Docs#Survival-Regression but I cannot make heads or tails of how you are supposed to do the actual implementation.

I am trying to model the "survivability" of a customer for a business. Survivability of a customer is a label given to customers based on if a purchase was made in the last month. If a customer fails to make a purchase, they are considered dead/censured. The two factors I am taking into account are "number of times advertised to" and "amount of time spent on business website". Data is collected about the customer on a monthly basis.

Here is what my data looks like for two customers (CustA and CustB) over three monthly time periods:

val seqCust = Seq(
//Customer,Period,Censor,# of Ads,Amount of Time on Site
("CustA",1,0,4,2400),
("CustA",2,0,6,1800),
("CustA",3,1,2,600),
("CustB",1,0,2,2800),
("CustB",2,0,4,2100),
("CustB",3,0,3,1200)
)

I then want to transform it into something like this as the docs specify:

val dfCust = seqCust.map(cr=>(cr._2,cr._3,Vectors.dense(cr._4,cr._5)).toDF("label", "censor", "features")

So that my data now looks like this:

[1,0,[4,2400]],
[2,0,[6,1800]],
[3,1,[2,600]],
[1,0,[2,2800]],
[2,0,[4,2100]],
[3,0,[3,1200]]

And then do the following:

val quantileProbabilities = Array(0.3, 0.6)
val aft = new AFTSurvivalRegression()
 .setQuantileProbabilities(quantileProbabilities)
 .setQuantilesCol("quantiles")

val model = aft.fit(dfCust)

// Print the coefficients, intercept and scale parameter for AFT survival regression
println(s"Coefficients: $model.coefficients")
println(s"Intercept: $model.intercept")
println(s"Scale: $model.scale")
model.transform(dfCust).show(false)

But I do not understand:

1. Is this the correct way to model the data as per Scala's documentation?


2. How come I am not taking the customer ID into account anywhere?

scala apache-spark survival-analysis survival

share|improve this question

edited Mar 21 at 20:37

EliSquared

asked Mar 21 at 3:05

EliSquaredEliSquared

346516

val seqCust = Seq(
//Customer,Period,Censor,# of Ads,Amount of Time on Site
("CustA",1,0,4,2400),
("CustA",2,0,6,1800),
("CustA",3,1,2,600),
("CustB",1,0,2,2800),
("CustB",2,0,4,2100),
("CustB",3,0,3,1200)
)

val dfCust = seqCust.map(cr=>(cr._2,cr._3,Vectors.dense(cr._4,cr._5)).toDF("label", "censor", "features")

[1,0,[4,2400]],
[2,0,[6,1800]],
[3,1,[2,600]],
[1,0,[2,2800]],
[2,0,[4,2100]],
[3,0,[3,1200]]

val quantileProbabilities = Array(0.3, 0.6)
val aft = new AFTSurvivalRegression()
 .setQuantileProbabilities(quantileProbabilities)
 .setQuantilesCol("quantiles")

val model = aft.fit(dfCust)

// Print the coefficients, intercept and scale parameter for AFT survival regression
println(s"Coefficients: $model.coefficients")
println(s"Intercept: $model.intercept")
println(s"Scale: $model.scale")
model.transform(dfCust).show(false)

share|improve this question

edited Mar 21 at 20:37

EliSquared

asked Mar 21 at 3:05

EliSquaredEliSquared

346516

share|improve this question

edited Mar 21 at 20:37

EliSquared

asked Mar 21 at 3:05

EliSquaredEliSquared

346516

asked Mar 21 at 3:05

EliSquaredEliSquared

346516

asked Mar 21 at 3:05

EliSquaredEliSquared

346516