Styjun

9 hrs long transit in DEL

What happened after the end of the Truman Show?

Playing a fast but quiet Alberti bass

Metal that glows when near pieces of itself

The Lucky House

What allows us to use imaginary numbers?

From France west coast to Portugal via ship?

Why did St. Jerome use "virago" in Gen. 2:23?

"A y'vama acquires herself through chalitza", really?

Installing certbot - error - "nothing provides pyparsing"

Show two plots together: a two dimensional curve tangent to the maxima of a three dimensional plot

What kind of (probable) traffic accident might lead to the desctruction of only (!) the brain stem and cerebellum?

Can I submit a paper computer science conference using an alias if using my real name can cause legal trouble in my original country

What's the point of writing that I know will never be used or read?

How to add a table description to a longtable?

Meaning and structure of headline "Hair it is: A List of ..."

Did they show Truman doing private things (toilet, etc) when filming him for 24 hours, 7 days a week?

Adding things to bunches of things vs multiplication

Why don't politicians push for fossil fuel reduction by pointing out their scarcity?

Eric Andre had a dream

What security risks does exposing the size of the plaintext entail?

Did Wernher von Braun really have a "Saturn V painted as the V2"?

Unsolved Problems due to Lack of Computational Power

Why doesn't mathematics collapse down, even though humans quite often make mistakes in their proofs?

Dealing with data which have different sizes in multi task learning (Pytorch)

Iterate over two list of different sizes in pythonabout torch.nn.CrossEntropyLoss parameter shapemulti-variable linear regression with pytorchIs this a correct reimplementation of Pytorch Seq2Seq model?PyTorch 0.4 LSTM: Why does each epoch get slower?Deep learning models with different input size?Multi-task learning: keep same batch size or same epochs?pytorch restore model with different batch sizePytorch RNN always gives the same output for multivariate time seriesLoss function used for multi-dimensional feature mappingWhat is the difference between MLP implementation from scratch and in PyTorch?

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

0

I want to create multi-task learning model with Pytorch.

The model I designed follows hard parameter sharing (http://ruder.io/multi-task/)

The problem is the size of each dataset is different. So they cannot be trained in the same for loop in the model like this:

for epoch in range(EPOCHS):
 for data in dataloader_train:
 output_a = model(data.a)
 output_b = model(data.b)
 output_c = model(data.c)

 loss_a = criterion(output_a, data.target_a)
 loss_b = criterion(output_b, data.target_b)
 loss_c = criterion(output_c, data.target_c)

 optimizer.zero_grad()
 loss.backward([loss_a, loss_b, loss_c])
 optimizer.step()

I would like to know how to deal with this situation. I thought I can use zip with itertools.cycle (Iterate over two list of different sizes in python) but it may affect the model significantly because it changes the frequency of certain data especially located in the early index:

data.a data.b
a 1
b 2
c 3
a 4
b 5
c 1
a 2
b 3

On the other hand, if I regard it as just the differences between epochs of the data sets, it seems that it has no problem. E.g. data.a has 5 epochs and data.b has 3 epochs when EPOCHS == 15.

Or if there is no problem to design model like this:

for epoch in range(EPOCHS):
 for data, target in data_a:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_b:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_c:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

I think this will be the most simple solution. However, I don't know why but I got obsessed with the idea that I have to build the model like the former model.

All the codes above are pseudocode.

Thank you for the advice :)

for-loop deep-learning pytorch subsampling

share|improve this question

asked Mar 27 at 13:57

c0mu51c4r7c0mu51c4r7

1

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  It is not clear on why you cannot concatenate data_a, data_b and data_c ? What size is different ? is it only the number of training examples ?
  
  – Cedias
  Mar 27 at 20:28
  

  
  
  
  

add a comment | 

0

I want to create multi-task learning model with Pytorch.

The model I designed follows hard parameter sharing (http://ruder.io/multi-task/)

The problem is the size of each dataset is different. So they cannot be trained in the same for loop in the model like this:

for epoch in range(EPOCHS):
 for data in dataloader_train:
 output_a = model(data.a)
 output_b = model(data.b)
 output_c = model(data.c)

 loss_a = criterion(output_a, data.target_a)
 loss_b = criterion(output_b, data.target_b)
 loss_c = criterion(output_c, data.target_c)

 optimizer.zero_grad()
 loss.backward([loss_a, loss_b, loss_c])
 optimizer.step()

I would like to know how to deal with this situation. I thought I can use zip with itertools.cycle (Iterate over two list of different sizes in python) but it may affect the model significantly because it changes the frequency of certain data especially located in the early index:

data.a data.b
a 1
b 2
c 3
a 4
b 5
c 1
a 2
b 3

On the other hand, if I regard it as just the differences between epochs of the data sets, it seems that it has no problem. E.g. data.a has 5 epochs and data.b has 3 epochs when EPOCHS == 15.

Or if there is no problem to design model like this:

for epoch in range(EPOCHS):
 for data, target in data_a:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_b:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_c:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

I think this will be the most simple solution. However, I don't know why but I got obsessed with the idea that I have to build the model like the former model.

All the codes above are pseudocode.

Thank you for the advice :)

for-loop deep-learning pytorch subsampling

share|improve this question

asked Mar 27 at 13:57

c0mu51c4r7c0mu51c4r7

1

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  It is not clear on why you cannot concatenate data_a, data_b and data_c ? What size is different ? is it only the number of training examples ?
  
  – Cedias
  Mar 27 at 20:28
  

  
  
  
  

add a comment | 

I want to create multi-task learning model with Pytorch.

The model I designed follows hard parameter sharing (http://ruder.io/multi-task/)

The problem is the size of each dataset is different. So they cannot be trained in the same for loop in the model like this:

for epoch in range(EPOCHS):
 for data in dataloader_train:
 output_a = model(data.a)
 output_b = model(data.b)
 output_c = model(data.c)

 loss_a = criterion(output_a, data.target_a)
 loss_b = criterion(output_b, data.target_b)
 loss_c = criterion(output_c, data.target_c)

 optimizer.zero_grad()
 loss.backward([loss_a, loss_b, loss_c])
 optimizer.step()

I would like to know how to deal with this situation. I thought I can use zip with itertools.cycle (Iterate over two list of different sizes in python) but it may affect the model significantly because it changes the frequency of certain data especially located in the early index:

data.a data.b
a 1
b 2
c 3
a 4
b 5
c 1
a 2
b 3

On the other hand, if I regard it as just the differences between epochs of the data sets, it seems that it has no problem. E.g. data.a has 5 epochs and data.b has 3 epochs when EPOCHS == 15.

Or if there is no problem to design model like this:

for epoch in range(EPOCHS):
 for data, target in data_a:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_b:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_c:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

I think this will be the most simple solution. However, I don't know why but I got obsessed with the idea that I have to build the model like the former model.

All the codes above are pseudocode.

Thank you for the advice :)

for-loop deep-learning pytorch subsampling

share|improve this question

asked Mar 27 at 13:57

c0mu51c4r7c0mu51c4r7

1

for epoch in range(EPOCHS):
 for data in dataloader_train:
 output_a = model(data.a)
 output_b = model(data.b)
 output_c = model(data.c)

 loss_a = criterion(output_a, data.target_a)
 loss_b = criterion(output_b, data.target_b)
 loss_c = criterion(output_c, data.target_c)

 optimizer.zero_grad()
 loss.backward([loss_a, loss_b, loss_c])
 optimizer.step()

data.a data.b
a 1
b 2
c 3
a 4
b 5
c 1
a 2
b 3

for epoch in range(EPOCHS):
 for data, target in data_a:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_b:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

 for data, target in data_c:
 output = model(data)
 loss = criterion(output, target)
 optimizer.zero_grad()
 loss.backward()
 optimizer.step()

share|improve this question

asked Mar 27 at 13:57

c0mu51c4r7c0mu51c4r7

1

share|improve this question

asked Mar 27 at 13:57

c0mu51c4r7c0mu51c4r7

1

asked Mar 27 at 13:57

c0mu51c4r7c0mu51c4r7

1

asked Mar 27 at 13:57

c0mu51c4r7c0mu51c4r7

1