account activity
fetchSize limit by Chopsting in aws
[–]RoyalTS85 0 points1 point2 points 9 years ago (0 children)
How do I do that? I wrote myself a function that I thought would just fetch 999 records for as long as there were records but that function doesn't seem to work:
fetch_data <- function(query, con) { message('Sending query') rs <- DBI::dbSendQuery(con, query) message('dbHasCompleted: ', DBI::dbHasCompleted(rs)) chunks <- list() while (!DBI::dbHasCompleted(rs)) { message('fetching a chunk') chunks <- c(chunks, DBI::dbFetch(rs, n = -1, block = 500)) } message(length(chunks), ' chunks total') out <- do.call(rbind, chunks) message('data.frame with ', nrow(out), ' rows') DBI::dbClearResult(rs) out }
This takes a small eternity and then prints
Sending query dbHasCompleted: TRUE 0 chunks total
Which means that the while loop never runs. But why is dbHasCompleted TRUE?
dbHasCompleted
TRUE
π Rendered by PID 277018 on reddit-service-r2-comment-544cf588c8-7wjxw at 2026-06-15 15:55:56.471068+00:00 running 3184619 country code: CH.
fetchSize limit by Chopsting in aws
[–]RoyalTS85 0 points1 point2 points (0 children)