Call Log totalPages Element Not Present

  • 1
  • Idea
  • Updated 2 years ago
  • Not Planned
I am attempting to pull all of our existing call-logs for a day, but we have more than 1000 logs that I need to pull. I have been using the totalPages element to iterate through all the pages. All other calls that I have used have this element (extensions, phone-number, presence, message-store, etc.). However, it appears that the call-log request does not return a totalPages element, making it difficult to accomplish this. I could continue to get the next page until I get no records, but this would require an extra call each time, and simply is not as clean of a soluton. This would also be inconsistent to what I am currently doing for all other calls.

Could you please let me know if I am missing something, or if there is a reason why this element does not exist for this call? This is very frustrating to run into seemingly random inconsistencies in your API.

Thanks for your time.
Photo of Spring IT

Spring IT

  • 310 Points 250 badge 2x thumb
  • Frustrated.

Posted 2 years ago

  • 1
Photo of Tyler Long

Tyler Long, Official Rep

  • 8,162 Points 5k badge 2x thumb
Confirmed and I agree it's inconsistent. We will switch this topic type to be an "idea" instead of a problem so it can act as a suggestion for product management.

Workarounds:

1. check `navigation.nextPage` element. If it doesn't exist, you don't need to issue an extra call
2. specify `perPage` query parameter to a large number. Then you might be able to get all the call logs with merely one call.
(Edited)
Photo of Spring IT

Spring IT

  • 310 Points 250 badge 2x thumb
Thanks for the reply!

It sounds like I will need to use the nextPage element, as all calls do not contain the totalPages/totalElements elements, and this wont be added to those that don't contain them from Anton's reply below. This is unfortunate, but a reality. Your second solution may work for some, but assuming that you'll never have more than 1000 records returned does not sound like a good idea.

Anyway, thanks again!
Photo of Anton Nikitin

Anton Nikitin, Official Rep

  • 2,914 Points 2k badge 2x thumb
It is not a bug or something we wish to fix going forward. totalPages and totalElements cannot be calculated efficiently for call logs since it will require fetching potentially unlimited number of elements in advance (just to calculate count) and has dramatic impact on server performance. So clients should work with dataset only using prev/next navigation which is pretty standard way to iterate through large datasets. 

I do not fully understand the issue. If a client needs to get all records, it will need to iterate anyway through all pages and the number of requests cannot be optimized. "perPage" parameter can be set to "max" value to work with maximum allowed page size.
Photo of Spring IT

Spring IT

  • 310 Points 250 badge 2x thumb
Thanks for the reply!

I see what you're saying. However, while I do not understand how your data is stored on the back end, in any traditional RDBMS (SQL Server, MySQL, etc.) getting a count of all elements requires substantially less processing then retrieving all of the records. If your data is stored in this way, it should be really easy, and fast to get a count of all of the records. Also, there should always be a date range for the logs that are being requested, and even if there was no date, I believe that you do not keep call logs indefinitely, so there will always be a cutoff for the amount of call logs available to pull. So saying that there is a potential to fetch an unlimited number of records is not right.

As far as I can tell, the max number of elements allowed on one page is 1000. This seems like a lot, but we have more than 1000 messages for any given location/day on a very regular basis. 

The reason that I chose to use the totalPages element to iterate through all pages was because I assumed (falsely) that all requests/results would be consistent, thus returning the totalPages for any and all calls. Secondly, using a specific number for iterations is a more concrete, predictable way to get all record. Checking for the nextPage element each and every time just requires a bit more processing.
Photo of Benjamin Dean

Benjamin Dean, Alum

  • 8,642 Points 5k badge 2x thumb
Actually, checking for nextPage element shouldn't be any more expensive.

Here is a recursive example using the JS SDK to fetch all call logs. It expects some environment variables to be set (which are prefixed with process.env.[ENV_VAR_NAME].

    function getCallLogsPage() {
        return platform
            .get('/account/~/call-log', {
                page: page,
                perPage: process.env.LOGS_PER_PAGE, // MAX is 1000
                dateFrom: process.env.DATE_FROM,
                dateTo: process.env.DATE_TO,
                view: process.env.VIEW
            })  
            .then(function (response) {
                console.log(response.json());
                // Sanity check
                if( response ) { 
                    console.log("************** SHOULD BE A SUCCESSFUL 200 REQUEST - NOTHING TO SEE HERE...MOVE ALONG PLEASE ***************");
                    if(timeoutObject) {
                        clearTimeout(timeoutObject);
                    }   
                    retryAfter = 0;
                    var data = response.json();
                    //console.log("************** THE NUMBER OF CALL LOGS THIS REQUEST CYCLE ARE : ***************", data.records.length);
                    callLogs = callLogs.concat(data.records);
                    if (data.navigation.nextPage) {
                        page++;
                        return getCallLogsPage(); // this will be chained
                    } else {
                        return callLogs; // this is the finally resolved thing
                    }
                }
            });
    } // end of getCallLogsPage()
    /*
     Loop until you capture all extensions, and then continue processing data
     */
    return getCallLogsPage()
    .then(<NEXT_FUNCTION_IN_PROGRAM>)
    .catch(function (e) {
            console.error(e);
            throw e;
        }
    });
Photo of Spring IT

Spring IT

  • 310 Points 250 badge 2x thumb
Thanks for your reply. I have my app updated to use the nextPage element. Thanks!
Photo of Spring IT

Spring IT

  • 310 Points 250 badge 2x thumb
I do believe that the API should be updated to return the totalPages/totalElements element for the call_logs, and any other calls that may be missing this. It does not makes sense to omit this from the one call, creating a discrepancy and preventing your users from being able to consistently use your API. It will only continue to cause your paying customers headache.

However, I do appreciate the quick reply and information.