How to print 1-10,11-20 and so on number of rows of a file in loop using shell? [closed]How do I iterate over a range of numbers defined by variables in Bash?How do I prompt for Yes/No/Cancel input in a Linux shell script?How to use SSH to run a shell script on a remote machine?Shell command to tar directory excluding certain files/foldersDiff files present in two different directoriesHow to echo shell commands as they are executedHow to declare and use boolean variables in shell script?How can I recursively find all files in current and subfolders based on wildcard matching?How to call shell script from another shell script?Replace whole line containing a string using Sed
Defense against attacks using dictionaries
How to use "Du hast/ Du hattest'?
Science fiction short story where aliens contact a drunk about Earth's impending destruction
Are modern clipless shoes and pedals that much better than toe clips and straps?
Was Switzerland really impossible to invade during WW2?
Why did MS-DOS applications built using Turbo Pascal fail to start with a division by zero error on faster systems?
What professions would a medieval village with a population of 100 need?
What are some interesting features that are common cross-linguistically but don't exist in English?
Is a player able to change alignment midway through an adventure?
What brought these couples together?
How much code would a codegolf golf if a codegolf could golf code?
How would one country purchase another?
Deck Size Problems in Deckbuilding
Efficiently pathfinding many flocking enemies around obstacles
Does travel insurance for short flight delays exist?
Accent on í misaligned in bibliography / citation
Middle Mouse turns on Perspective
Why do all fields in a QFT transform like *irreducible* representations of some group?
Can you feel passing through the sound barrier in an F-16?
Sun setting in East!
Did a flight controller ever answer Flight with a no-go?
Is there any practical application for performing a double Fourier transform? ...or an inverse Fourier transform on a time-domain input?
In the MCU, why does Mjölnir retain its enchantments after Ragnarok?
What does どうかと思う mean?
How to print 1-10,11-20 and so on number of rows of a file in loop using shell? [closed]
How do I iterate over a range of numbers defined by variables in Bash?How do I prompt for Yes/No/Cancel input in a Linux shell script?How to use SSH to run a shell script on a remote machine?Shell command to tar directory excluding certain files/foldersDiff files present in two different directoriesHow to echo shell commands as they are executedHow to declare and use boolean variables in shell script?How can I recursively find all files in current and subfolders based on wildcard matching?How to call shell script from another shell script?Replace whole line containing a string using Sed
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have a file consisting of 4000 rows, I need to iterate the records of that file over shell script and extract first 10 rows and send that rows to my java code which i already wrote, and then next 10 rows and so on
shell awk sed
closed as unclear what you're asking by Ed Morton, jww, Max Vollmer, Serenity, Sinto Apr 1 at 4:12
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
I have a file consisting of 4000 rows, I need to iterate the records of that file over shell script and extract first 10 rows and send that rows to my java code which i already wrote, and then next 10 rows and so on
shell awk sed
closed as unclear what you're asking by Ed Morton, jww, Max Vollmer, Serenity, Sinto Apr 1 at 4:12
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
3
edit your question to clairfy: When you saysend that rows
- do you mean pipe them or pass them as arguments or open some kind of ssh channel or something else? At least show how your java script would be called with one instance of 10 rows.
– Ed Morton
Mar 27 at 17:04
What shell are you using?
– bishop
Mar 27 at 17:14
add a comment |
I have a file consisting of 4000 rows, I need to iterate the records of that file over shell script and extract first 10 rows and send that rows to my java code which i already wrote, and then next 10 rows and so on
shell awk sed
I have a file consisting of 4000 rows, I need to iterate the records of that file over shell script and extract first 10 rows and send that rows to my java code which i already wrote, and then next 10 rows and so on
shell awk sed
shell awk sed
asked Mar 27 at 16:44
Muddassir RahmanMuddassir Rahman
2882 silver badges12 bronze badges
2882 silver badges12 bronze badges
closed as unclear what you're asking by Ed Morton, jww, Max Vollmer, Serenity, Sinto Apr 1 at 4:12
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
closed as unclear what you're asking by Ed Morton, jww, Max Vollmer, Serenity, Sinto Apr 1 at 4:12
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
closed as unclear what you're asking by Ed Morton, jww, Max Vollmer, Serenity, Sinto Apr 1 at 4:12
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
3
edit your question to clairfy: When you saysend that rows
- do you mean pipe them or pass them as arguments or open some kind of ssh channel or something else? At least show how your java script would be called with one instance of 10 rows.
– Ed Morton
Mar 27 at 17:04
What shell are you using?
– bishop
Mar 27 at 17:14
add a comment |
3
edit your question to clairfy: When you saysend that rows
- do you mean pipe them or pass them as arguments or open some kind of ssh channel or something else? At least show how your java script would be called with one instance of 10 rows.
– Ed Morton
Mar 27 at 17:04
What shell are you using?
– bishop
Mar 27 at 17:14
3
3
edit your question to clairfy: When you say
send that rows
- do you mean pipe them or pass them as arguments or open some kind of ssh channel or something else? At least show how your java script would be called with one instance of 10 rows.– Ed Morton
Mar 27 at 17:04
edit your question to clairfy: When you say
send that rows
- do you mean pipe them or pass them as arguments or open some kind of ssh channel or something else? At least show how your java script would be called with one instance of 10 rows.– Ed Morton
Mar 27 at 17:04
What shell are you using?
– bishop
Mar 27 at 17:14
What shell are you using?
– bishop
Mar 27 at 17:14
add a comment |
6 Answers
6
active
oldest
votes
To pass 10 lines at a time as arguments to your script:
< file xargs -d$'n' -n 10 myscript
To pipe 10 lines at a time as input to your script:
< file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | myscript'
Assuming your input is in a file named file
which I'm creating with 30 instead of 4000 lines of input:
$ seq 30 > file
and modifying to have some lines that contain spaces, some that contain shell variables, and some that contain regexp and globbing chars to show no type of shell expansion is being done:
$ head -10 file
1
here is a multi-field line
3
4
$HOME
6
.*
8
9
10
Here's 10 args at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 awk 'BEGINfor (i=1; i<ARGC; i++) print i, "<" ARGV[i] ">"; exit ENDprint "---"'
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
and here's 10 lines of input at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | awk '''print NR, "<" $0 ">" ENDprint "---"''''
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
Can you explain sir plz
– Muddassir Rahman
Mar 27 at 16:53
2
What part of it don't you understand after reading thexargs
man page?
– Ed Morton
Mar 27 at 16:54
add a comment |
Considering that OP wants to pass lines as an argument to OP's code if that is the case then could you please try following once(haven't tested it by running it since I don't have OP's java code etc).
awk '
FNR%10==0
system("your_java_code " value OFS $0)
value=""
value=(value?value OFS:"")$0
END
if(value)
system("your_java_code " value)
' Input_file
OR
awk '
value=(value?value OFS:"")$0
FNR%10==0
system("your_java_code " value)
value=""
END
if(value)
system("your_java_code " value)
' Input_file
PS: Just for safer side, I kept END
section of awk
code so that in case there are left over lines(let's say total number of lines are NOT completely divided by 10) then it will call java program with remaining lines to it.
add a comment |
Sounds to me like you want to slice out rows from a file, then pipe those rows to java. This interpretation differs from the other answers, so let me know if I'm not understanding you:
$ file=/etc/services
$ count=$(wc -l < "$file")
$ start=1
$ stride=10
$ for ((i=start; i<=count; i+=stride)); do
awk -v i="$i" -v stride="$stride"
'NR > (i+stride) exit NR >= i && NR < (i + stride)' "$file"
| java ...
done
file
holds the path to the data rows. count
is the total count of rows in that file. start
is the first row, stride
is how many you want to slice out in each iteration.
The for loop then performs the stride addition, while awk
slices out the rows so numbered. We pipe them to the java
program on standard in.
add a comment |
Assuming that you are passing the 10 lines groups from your file to your script as command line arguments, this is an answer:
rows=4000 # the number of rows in file
groupsize=10 # the size of lines groups
OIFS="$IFS"; IFS=$'n' # use newline as input field separator to avoid `for` splitting on spaces
groups=$(($rows / $groupsize)) # the number of groups of lines
for i in $(seq 1 $groups); do # loop through each group of lines
from=$((($i * $groupsize) - $groupsize + 1))
to=$(($i * $groupsize))
# build the arguments for each script invocation by concatenating each group of lines
for line in `sed -n -e $from,$top file`; do # 'file' is your input file name
arguments=$arguments "$line"
done
echo script $arguments # remove echo and change 'script' with your script name
done
IFS="$OIFS" # restore original input field separator
add a comment |
Like this :
for ((i=0; i<=4000; i+=10)); do
arr=( ) # create a new empty array
for ((j=$i; j<=i+10; j++)); do
arr+=( $j ) # add id to array
done
printf '%sn' "$arr[@]" # or execute command with all the id
done
Can you explain little bit plz
– Muddassir Rahman
Mar 27 at 16:50
1
It's just arithmetic
– Gilles Quenot
Mar 27 at 16:55
Added some comments
– Gilles Quenot
Mar 27 at 21:10
add a comment |
This might work for you (GNU parallel):
parallel -kN10 javaProgram :::: file
This will pass the lines 1-10, 11-20, ... as arguments to program javaProgram
If you want to pass 10 lines at time, use:
parallel -kN10 --cat javaProgram :::: file
add a comment |
6 Answers
6
active
oldest
votes
6 Answers
6
active
oldest
votes
active
oldest
votes
active
oldest
votes
To pass 10 lines at a time as arguments to your script:
< file xargs -d$'n' -n 10 myscript
To pipe 10 lines at a time as input to your script:
< file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | myscript'
Assuming your input is in a file named file
which I'm creating with 30 instead of 4000 lines of input:
$ seq 30 > file
and modifying to have some lines that contain spaces, some that contain shell variables, and some that contain regexp and globbing chars to show no type of shell expansion is being done:
$ head -10 file
1
here is a multi-field line
3
4
$HOME
6
.*
8
9
10
Here's 10 args at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 awk 'BEGINfor (i=1; i<ARGC; i++) print i, "<" ARGV[i] ">"; exit ENDprint "---"'
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
and here's 10 lines of input at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | awk '''print NR, "<" $0 ">" ENDprint "---"''''
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
Can you explain sir plz
– Muddassir Rahman
Mar 27 at 16:53
2
What part of it don't you understand after reading thexargs
man page?
– Ed Morton
Mar 27 at 16:54
add a comment |
To pass 10 lines at a time as arguments to your script:
< file xargs -d$'n' -n 10 myscript
To pipe 10 lines at a time as input to your script:
< file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | myscript'
Assuming your input is in a file named file
which I'm creating with 30 instead of 4000 lines of input:
$ seq 30 > file
and modifying to have some lines that contain spaces, some that contain shell variables, and some that contain regexp and globbing chars to show no type of shell expansion is being done:
$ head -10 file
1
here is a multi-field line
3
4
$HOME
6
.*
8
9
10
Here's 10 args at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 awk 'BEGINfor (i=1; i<ARGC; i++) print i, "<" ARGV[i] ">"; exit ENDprint "---"'
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
and here's 10 lines of input at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | awk '''print NR, "<" $0 ">" ENDprint "---"''''
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
Can you explain sir plz
– Muddassir Rahman
Mar 27 at 16:53
2
What part of it don't you understand after reading thexargs
man page?
– Ed Morton
Mar 27 at 16:54
add a comment |
To pass 10 lines at a time as arguments to your script:
< file xargs -d$'n' -n 10 myscript
To pipe 10 lines at a time as input to your script:
< file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | myscript'
Assuming your input is in a file named file
which I'm creating with 30 instead of 4000 lines of input:
$ seq 30 > file
and modifying to have some lines that contain spaces, some that contain shell variables, and some that contain regexp and globbing chars to show no type of shell expansion is being done:
$ head -10 file
1
here is a multi-field line
3
4
$HOME
6
.*
8
9
10
Here's 10 args at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 awk 'BEGINfor (i=1; i<ARGC; i++) print i, "<" ARGV[i] ">"; exit ENDprint "---"'
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
and here's 10 lines of input at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | awk '''print NR, "<" $0 ">" ENDprint "---"''''
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
To pass 10 lines at a time as arguments to your script:
< file xargs -d$'n' -n 10 myscript
To pipe 10 lines at a time as input to your script:
< file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | myscript'
Assuming your input is in a file named file
which I'm creating with 30 instead of 4000 lines of input:
$ seq 30 > file
and modifying to have some lines that contain spaces, some that contain shell variables, and some that contain regexp and globbing chars to show no type of shell expansion is being done:
$ head -10 file
1
here is a multi-field line
3
4
$HOME
6
.*
8
9
10
Here's 10 args at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 awk 'BEGINfor (i=1; i<ARGC; i++) print i, "<" ARGV[i] ">"; exit ENDprint "---"'
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
and here's 10 lines of input at a time being passed to an awk script:
$ < file xargs -d$'n' -n 10 sh -c 'printf "%sn" "$@" | awk '''print NR, "<" $0 ">" ENDprint "---"''''
1 <1>
2 <here is a multi-field line>
3 <3>
4 <4>
5 <$HOME>
6 <6>
7 <.*>
8 <8>
9 <9>
10 <10>
---
1 <11>
2 <12>
3 <13>
4 <14>
5 <15>
6 <16>
7 <17>
8 <18>
9 <19>
10 <20>
---
1 <21>
2 <22>
3 <23>
4 <24>
5 <25>
6 <26>
7 <27>
8 <28>
9 <29>
10 <30>
---
edited Mar 28 at 13:23
answered Mar 27 at 16:51
Ed MortonEd Morton
124k13 gold badges48 silver badges108 bronze badges
124k13 gold badges48 silver badges108 bronze badges
Can you explain sir plz
– Muddassir Rahman
Mar 27 at 16:53
2
What part of it don't you understand after reading thexargs
man page?
– Ed Morton
Mar 27 at 16:54
add a comment |
Can you explain sir plz
– Muddassir Rahman
Mar 27 at 16:53
2
What part of it don't you understand after reading thexargs
man page?
– Ed Morton
Mar 27 at 16:54
Can you explain sir plz
– Muddassir Rahman
Mar 27 at 16:53
Can you explain sir plz
– Muddassir Rahman
Mar 27 at 16:53
2
2
What part of it don't you understand after reading the
xargs
man page?– Ed Morton
Mar 27 at 16:54
What part of it don't you understand after reading the
xargs
man page?– Ed Morton
Mar 27 at 16:54
add a comment |
Considering that OP wants to pass lines as an argument to OP's code if that is the case then could you please try following once(haven't tested it by running it since I don't have OP's java code etc).
awk '
FNR%10==0
system("your_java_code " value OFS $0)
value=""
value=(value?value OFS:"")$0
END
if(value)
system("your_java_code " value)
' Input_file
OR
awk '
value=(value?value OFS:"")$0
FNR%10==0
system("your_java_code " value)
value=""
END
if(value)
system("your_java_code " value)
' Input_file
PS: Just for safer side, I kept END
section of awk
code so that in case there are left over lines(let's say total number of lines are NOT completely divided by 10) then it will call java program with remaining lines to it.
add a comment |
Considering that OP wants to pass lines as an argument to OP's code if that is the case then could you please try following once(haven't tested it by running it since I don't have OP's java code etc).
awk '
FNR%10==0
system("your_java_code " value OFS $0)
value=""
value=(value?value OFS:"")$0
END
if(value)
system("your_java_code " value)
' Input_file
OR
awk '
value=(value?value OFS:"")$0
FNR%10==0
system("your_java_code " value)
value=""
END
if(value)
system("your_java_code " value)
' Input_file
PS: Just for safer side, I kept END
section of awk
code so that in case there are left over lines(let's say total number of lines are NOT completely divided by 10) then it will call java program with remaining lines to it.
add a comment |
Considering that OP wants to pass lines as an argument to OP's code if that is the case then could you please try following once(haven't tested it by running it since I don't have OP's java code etc).
awk '
FNR%10==0
system("your_java_code " value OFS $0)
value=""
value=(value?value OFS:"")$0
END
if(value)
system("your_java_code " value)
' Input_file
OR
awk '
value=(value?value OFS:"")$0
FNR%10==0
system("your_java_code " value)
value=""
END
if(value)
system("your_java_code " value)
' Input_file
PS: Just for safer side, I kept END
section of awk
code so that in case there are left over lines(let's say total number of lines are NOT completely divided by 10) then it will call java program with remaining lines to it.
Considering that OP wants to pass lines as an argument to OP's code if that is the case then could you please try following once(haven't tested it by running it since I don't have OP's java code etc).
awk '
FNR%10==0
system("your_java_code " value OFS $0)
value=""
value=(value?value OFS:"")$0
END
if(value)
system("your_java_code " value)
' Input_file
OR
awk '
value=(value?value OFS:"")$0
FNR%10==0
system("your_java_code " value)
value=""
END
if(value)
system("your_java_code " value)
' Input_file
PS: Just for safer side, I kept END
section of awk
code so that in case there are left over lines(let's say total number of lines are NOT completely divided by 10) then it will call java program with remaining lines to it.
edited Mar 27 at 17:12
answered Mar 27 at 17:06
RavinderSingh13RavinderSingh13
33.5k4 gold badges16 silver badges39 bronze badges
33.5k4 gold badges16 silver badges39 bronze badges
add a comment |
add a comment |
Sounds to me like you want to slice out rows from a file, then pipe those rows to java. This interpretation differs from the other answers, so let me know if I'm not understanding you:
$ file=/etc/services
$ count=$(wc -l < "$file")
$ start=1
$ stride=10
$ for ((i=start; i<=count; i+=stride)); do
awk -v i="$i" -v stride="$stride"
'NR > (i+stride) exit NR >= i && NR < (i + stride)' "$file"
| java ...
done
file
holds the path to the data rows. count
is the total count of rows in that file. start
is the first row, stride
is how many you want to slice out in each iteration.
The for loop then performs the stride addition, while awk
slices out the rows so numbered. We pipe them to the java
program on standard in.
add a comment |
Sounds to me like you want to slice out rows from a file, then pipe those rows to java. This interpretation differs from the other answers, so let me know if I'm not understanding you:
$ file=/etc/services
$ count=$(wc -l < "$file")
$ start=1
$ stride=10
$ for ((i=start; i<=count; i+=stride)); do
awk -v i="$i" -v stride="$stride"
'NR > (i+stride) exit NR >= i && NR < (i + stride)' "$file"
| java ...
done
file
holds the path to the data rows. count
is the total count of rows in that file. start
is the first row, stride
is how many you want to slice out in each iteration.
The for loop then performs the stride addition, while awk
slices out the rows so numbered. We pipe them to the java
program on standard in.
add a comment |
Sounds to me like you want to slice out rows from a file, then pipe those rows to java. This interpretation differs from the other answers, so let me know if I'm not understanding you:
$ file=/etc/services
$ count=$(wc -l < "$file")
$ start=1
$ stride=10
$ for ((i=start; i<=count; i+=stride)); do
awk -v i="$i" -v stride="$stride"
'NR > (i+stride) exit NR >= i && NR < (i + stride)' "$file"
| java ...
done
file
holds the path to the data rows. count
is the total count of rows in that file. start
is the first row, stride
is how many you want to slice out in each iteration.
The for loop then performs the stride addition, while awk
slices out the rows so numbered. We pipe them to the java
program on standard in.
Sounds to me like you want to slice out rows from a file, then pipe those rows to java. This interpretation differs from the other answers, so let me know if I'm not understanding you:
$ file=/etc/services
$ count=$(wc -l < "$file")
$ start=1
$ stride=10
$ for ((i=start; i<=count; i+=stride)); do
awk -v i="$i" -v stride="$stride"
'NR > (i+stride) exit NR >= i && NR < (i + stride)' "$file"
| java ...
done
file
holds the path to the data rows. count
is the total count of rows in that file. start
is the first row, stride
is how many you want to slice out in each iteration.
The for loop then performs the stride addition, while awk
slices out the rows so numbered. We pipe them to the java
program on standard in.
edited Mar 27 at 17:13
answered Mar 27 at 16:59
bishopbishop
26.2k6 gold badges71 silver badges105 bronze badges
26.2k6 gold badges71 silver badges105 bronze badges
add a comment |
add a comment |
Assuming that you are passing the 10 lines groups from your file to your script as command line arguments, this is an answer:
rows=4000 # the number of rows in file
groupsize=10 # the size of lines groups
OIFS="$IFS"; IFS=$'n' # use newline as input field separator to avoid `for` splitting on spaces
groups=$(($rows / $groupsize)) # the number of groups of lines
for i in $(seq 1 $groups); do # loop through each group of lines
from=$((($i * $groupsize) - $groupsize + 1))
to=$(($i * $groupsize))
# build the arguments for each script invocation by concatenating each group of lines
for line in `sed -n -e $from,$top file`; do # 'file' is your input file name
arguments=$arguments "$line"
done
echo script $arguments # remove echo and change 'script' with your script name
done
IFS="$OIFS" # restore original input field separator
add a comment |
Assuming that you are passing the 10 lines groups from your file to your script as command line arguments, this is an answer:
rows=4000 # the number of rows in file
groupsize=10 # the size of lines groups
OIFS="$IFS"; IFS=$'n' # use newline as input field separator to avoid `for` splitting on spaces
groups=$(($rows / $groupsize)) # the number of groups of lines
for i in $(seq 1 $groups); do # loop through each group of lines
from=$((($i * $groupsize) - $groupsize + 1))
to=$(($i * $groupsize))
# build the arguments for each script invocation by concatenating each group of lines
for line in `sed -n -e $from,$top file`; do # 'file' is your input file name
arguments=$arguments "$line"
done
echo script $arguments # remove echo and change 'script' with your script name
done
IFS="$OIFS" # restore original input field separator
add a comment |
Assuming that you are passing the 10 lines groups from your file to your script as command line arguments, this is an answer:
rows=4000 # the number of rows in file
groupsize=10 # the size of lines groups
OIFS="$IFS"; IFS=$'n' # use newline as input field separator to avoid `for` splitting on spaces
groups=$(($rows / $groupsize)) # the number of groups of lines
for i in $(seq 1 $groups); do # loop through each group of lines
from=$((($i * $groupsize) - $groupsize + 1))
to=$(($i * $groupsize))
# build the arguments for each script invocation by concatenating each group of lines
for line in `sed -n -e $from,$top file`; do # 'file' is your input file name
arguments=$arguments "$line"
done
echo script $arguments # remove echo and change 'script' with your script name
done
IFS="$OIFS" # restore original input field separator
Assuming that you are passing the 10 lines groups from your file to your script as command line arguments, this is an answer:
rows=4000 # the number of rows in file
groupsize=10 # the size of lines groups
OIFS="$IFS"; IFS=$'n' # use newline as input field separator to avoid `for` splitting on spaces
groups=$(($rows / $groupsize)) # the number of groups of lines
for i in $(seq 1 $groups); do # loop through each group of lines
from=$((($i * $groupsize) - $groupsize + 1))
to=$(($i * $groupsize))
# build the arguments for each script invocation by concatenating each group of lines
for line in `sed -n -e $from,$top file`; do # 'file' is your input file name
arguments=$arguments "$line"
done
echo script $arguments # remove echo and change 'script' with your script name
done
IFS="$OIFS" # restore original input field separator
edited Mar 27 at 17:31
answered Mar 27 at 17:21
MarcoSMarcoS
10.5k18 gold badges71 silver badges141 bronze badges
10.5k18 gold badges71 silver badges141 bronze badges
add a comment |
add a comment |
Like this :
for ((i=0; i<=4000; i+=10)); do
arr=( ) # create a new empty array
for ((j=$i; j<=i+10; j++)); do
arr+=( $j ) # add id to array
done
printf '%sn' "$arr[@]" # or execute command with all the id
done
Can you explain little bit plz
– Muddassir Rahman
Mar 27 at 16:50
1
It's just arithmetic
– Gilles Quenot
Mar 27 at 16:55
Added some comments
– Gilles Quenot
Mar 27 at 21:10
add a comment |
Like this :
for ((i=0; i<=4000; i+=10)); do
arr=( ) # create a new empty array
for ((j=$i; j<=i+10; j++)); do
arr+=( $j ) # add id to array
done
printf '%sn' "$arr[@]" # or execute command with all the id
done
Can you explain little bit plz
– Muddassir Rahman
Mar 27 at 16:50
1
It's just arithmetic
– Gilles Quenot
Mar 27 at 16:55
Added some comments
– Gilles Quenot
Mar 27 at 21:10
add a comment |
Like this :
for ((i=0; i<=4000; i+=10)); do
arr=( ) # create a new empty array
for ((j=$i; j<=i+10; j++)); do
arr+=( $j ) # add id to array
done
printf '%sn' "$arr[@]" # or execute command with all the id
done
Like this :
for ((i=0; i<=4000; i+=10)); do
arr=( ) # create a new empty array
for ((j=$i; j<=i+10; j++)); do
arr+=( $j ) # add id to array
done
printf '%sn' "$arr[@]" # or execute command with all the id
done
edited Mar 27 at 21:10
answered Mar 27 at 16:48
Gilles QuenotGilles Quenot
114k23 gold badges171 silver badges174 bronze badges
114k23 gold badges171 silver badges174 bronze badges
Can you explain little bit plz
– Muddassir Rahman
Mar 27 at 16:50
1
It's just arithmetic
– Gilles Quenot
Mar 27 at 16:55
Added some comments
– Gilles Quenot
Mar 27 at 21:10
add a comment |
Can you explain little bit plz
– Muddassir Rahman
Mar 27 at 16:50
1
It's just arithmetic
– Gilles Quenot
Mar 27 at 16:55
Added some comments
– Gilles Quenot
Mar 27 at 21:10
Can you explain little bit plz
– Muddassir Rahman
Mar 27 at 16:50
Can you explain little bit plz
– Muddassir Rahman
Mar 27 at 16:50
1
1
It's just arithmetic
– Gilles Quenot
Mar 27 at 16:55
It's just arithmetic
– Gilles Quenot
Mar 27 at 16:55
Added some comments
– Gilles Quenot
Mar 27 at 21:10
Added some comments
– Gilles Quenot
Mar 27 at 21:10
add a comment |
This might work for you (GNU parallel):
parallel -kN10 javaProgram :::: file
This will pass the lines 1-10, 11-20, ... as arguments to program javaProgram
If you want to pass 10 lines at time, use:
parallel -kN10 --cat javaProgram :::: file
add a comment |
This might work for you (GNU parallel):
parallel -kN10 javaProgram :::: file
This will pass the lines 1-10, 11-20, ... as arguments to program javaProgram
If you want to pass 10 lines at time, use:
parallel -kN10 --cat javaProgram :::: file
add a comment |
This might work for you (GNU parallel):
parallel -kN10 javaProgram :::: file
This will pass the lines 1-10, 11-20, ... as arguments to program javaProgram
If you want to pass 10 lines at time, use:
parallel -kN10 --cat javaProgram :::: file
This might work for you (GNU parallel):
parallel -kN10 javaProgram :::: file
This will pass the lines 1-10, 11-20, ... as arguments to program javaProgram
If you want to pass 10 lines at time, use:
parallel -kN10 --cat javaProgram :::: file
edited Mar 28 at 10:59
answered Mar 28 at 10:53
potongpotong
38.3k4 gold badges33 silver badges64 bronze badges
38.3k4 gold badges33 silver badges64 bronze badges
add a comment |
add a comment |
3
edit your question to clairfy: When you say
send that rows
- do you mean pipe them or pass them as arguments or open some kind of ssh channel or something else? At least show how your java script would be called with one instance of 10 rows.– Ed Morton
Mar 27 at 17:04
What shell are you using?
– bishop
Mar 27 at 17:14