Styjun

I am developing an application and I encountered some difficulties in the following situation.

I have an url that at its end accepts a pageSize and the current page, and what I am trying to do is using that URL, get in a list all the elements from a database table, without knowing how many pages I actually have (could be one page of data or could be 1 million pages).

For example, after I make a request with lets say page size 50 and page 1, I want to receive page 2, and if page 2 has 50 elements in it, to receive page 3, if it has anything less than 50 elements to stop there and return in the end all the data I received until now, and nothing to be returned until I made a last request.

What I tried to do is the following thing:

 return this.http.get(url + currentPage, headers ).pipe(
 mergeMap((response: any) => 
 managedObject = managedObject.concat(response.managedObjects);
 if (response.managedObjects.length === pageSizeNo) 
 currentPage++;
 this.http.get(url + currentPage, headers );
 else 
 return managedObject;
 
 )
);

I initially give the url with pageSizeNo = 50 defined in it and currentPage = 1, then after I receive the response I want to go further and make another call or return everything I received until now. The problem is that everything stops at page 2, and no further call happens.

This is how I subscribe to what the function returns:

this.service.fetchData()
 .subscribe((response: any) => 
 this.qualityData = response
);

How can I fix this?

Use expand operator to recursively call http-get. E.g.:

return this.http.get(url + currentPage, headers ).pipe(
 expand((response: any) => 
 managedObject = managedObject.concat(response.managedObjects);
 if (managedObject.length === pageSizeNo) 
 currentPage++;
 return this.http.get(url + currentPage, headers );
 else 
 return EMPTY;
 
 ),
 // combine all emitted values into one array
 toArray()
)

Try this expand operator example

NOTE: currentPage might leak, so be careful with it.

function getPage(pageNo: number): Observable< managedObjects: any[] > 
 return this.http.get(`url?page=$pageNo`, headers );


of([0, false, []] as [number, boolean, any[]]).pipe(
 expand(([pageNo, isFinal, allResults]) => getPage(pageNo).pipe(
 map(newResults => [
 pageNo + 1,
 newResults.managedObjects.length < 50,
 [...allResults, ...newResults.managedObjects]
 ] as [number, boolean, any[]])
 )),
 takeWhile(([pageNo, isFinal, allResults]) => !isFinal, true),
 last(),
 map(([pageNo, isFinal, allResults]) => allResults)
).subscribe(finalResults => 
 // ...
);

See: https://stackblitz.com/edit/rxjs-tyuefr?file=index.ts for a working example