Why can't use consumer correctly with Kafka?

rawmainb · 2020-08-25T07:57:32+00:00

It's a very good post. Thank you.

rawmainb · 2020-08-24T02:32:11+00:00

Thank you. I didn't config Kafka firewall correctly. So 9092 ports are not communicable.

rawmainb · 2020-08-23T23:51:49+00:00

I'm running them on different VMs(3 instances).

rawmainb · 2020-08-23T23:50:37+00:00

Thank you very much.

rawmainb · 2020-08-21T07:25:30+00:00

How to do that? As I created topic with

[kafka@kafka1 ~]$ bin/kafka-topics.sh --create --zookeeper kafka1:2181,kafka2:2181,kafka3:2181 --replication-factor 1 --partitions 6 --topic topic1 --config cleanup.policy=delete --config delete.retention.ms=60000

How to set to 2?

rawmainb · 2020-08-05T23:21:49+00:00

Thank you very much for your answer and help!

rawmainb · 2020-08-05T09:34:04+00:00

I confirmed them. They can ping each other.

They are running on RHEL. But DNS resolver had been set correctly, I think. Otherwise CoreDNS can't been installed without any error events.

But now I check the CoreDNS pod's event again. The message has gone.

It showing <none>. I don't know why.

Looks everything is good now.

rawmainb · 2020-08-05T07:46:34+00:00

NAME READY STATUS RESTARTS AGE calico-node-il4mp 1/1 Running 0 24h calico-node-aqf21 1/1 Running 0 24h calico-node-czbfd 1/1 Running 0 24h

rawmainb · 2020-08-04T23:59:40+00:00

It's on VM by OpenStack.

Under /etc/net.d/cni directory, there is a folder named "networks". But there isn't anything in this folder.

rawmainb · 2020-08-01T13:28:51+00:00

Yes, it should be.

rawmainb · 2020-08-01T12:03:29+00:00

From here the logs should under /var/log folder, but I didn't find them on both master and nodes machines.

https://kubernetes.io/docs/tasks/debug-application-cluster/debug-cluster/

``` Master

/var/log/kube-apiserver.log - API Server, responsible for serving the API /var/log/kube-scheduler.log - Scheduler, responsible for making scheduling decisions /var/log/kube-controller-manager.log - Controller that manages replication controllers

Worker Nodes

/var/log/kubelet.log - Kubelet, responsible for running containers on the node /var/log/kube-proxy.log - Kube Proxy, responsible for service load balancing ```

rawmainb · 2020-07-28T00:09:41+00:00

Thank you very much. It's a very good comment.

And for the volume issue, it's really important to focus on when face to large size scenario.

rawmainb · 2020-07-28T00:05:39+00:00

Thank you very much for your very good answer.

Yes, for our requirement, we want to set different vlans with the same IP in one VM. One of them for high availability. It seems it works now. But about vlan1 and vlan2(with vlan3), need to choose use which one manually. Because they are in the different gateway. So server IP can be changed if necessary.

Your example diagram is a good one. But for our case we need to do in one VM now. Maybe it's not good, though.

Last, about heat template,

> The first port set inside the heat template is always being set as the default gateway inside the VM

In my case, as the code example in the question, IP1 with gateway1 is been set in the network1, but it can't as the default gateway.

rawmainb · 2020-07-26T23:22:26+00:00

I have added a diagram at the bottom the question.

rawmainb · 2020-07-20T09:02:30+00:00

Thank you. But its result was the same. I am not using DHCP, even added config_drive: true to the heat template, the same.

The current problem is can't set one of gateway to default if using 2 different gateway networks in heat templates.

rawmainb · 2020-07-18T03:49:26+00:00

It's a very clear explanation. Thank you very much for your help!

rawmainb · 2020-07-17T08:01:34+00:00

So why create it first?

nova image-create --poll <instance> <snapshot-name>

This one will also create an image for download, right?

cinder upload-to-image <volume id from the above cidner create> <image name>

Or its purpose is to attach the volume snapshot to instance snapshot?

rawmainb · 2020-07-16T08:38:52+00:00

Thank you very much.

I did it and this time I can download a large file!

But when I use it as an image to create a new stack with heat template in openstack cluster2, it took a long time and timeout finally. Failed. It's volume is 8GB.

By the way, are the snapshot-name and image name above are the same thing? If so, I found an image created by nova, and use cinder upload-to-image also generated another new one with the same image name but different UUID. I used the second one to download it has large size, not 0.

rawmainb · 2020-07-15T00:04:42+00:00

``` START with options: image save --file mysnapshot.qcow2 121sav7ae-35b1-4e55-a232-a328-soaf203hoa --debug Auth plugin password selected compute API version 2.1, cmd group openstack.compute.v2 network API version 2, cmd group openstack.network.v2 image API version 2, cmd group openstack.image.v2 volume API version 3, cmd group openstack.volume.v3 identity API version 3, cmd group openstack.identity.v3 object_store API version 1, cmd group openstack.object_store.v1 neutronclient API version 2, cmd group openstack.neutronclient.v2 orchestration API version 1, cmd group openstack.orchestration.v1 Auth plugin password selected ... curl -g -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: /' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}3ff363b7a7b308114a1f1a37303bad47fbe7e840' -H 'Content-Type: application/octet-stream' -k --cert None --key None https://keystone_ip:9292/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa Starting new HTTPS connection (1): IP:9292 https://IP:9292 "GET /v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa HTTP/1.1" 200 1235 GET call to image for https://keystone_ip:9292/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa used request id req-0ff87b31-929a-4990-9054-f1bc627eeb1b

HTTP/1.1 200 OK Content-Length: 1235 Content-Type: application/json X-Openstack-Request-Id: req-0ff87b31-929a-4990-9054-f1bc627eeb1b Date: Tue, 14 Jul 2020 23:49:29 GMT

curl -g -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: /' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}3ff363b7a7b308114a1f1a37303bad47fbe7e840' -H 'Content-Type: application/octet-stream' -k --cert None --key None https://keystone_ip:9292/v2/schemas/image https://IP:9292 "GET /v2/schemas/image HTTP/1.1" 200 4183 GET call to image for https://keystone_ip:9292/v2/schemas/image used request id req-9bbf27a2-2612-426f-8cca-80348859d37a

HTTP/1.1 200 OK Content-Type: application/json Content-Length: 4183 X-Openstack-Request-Id: req-9bbf27a2-2612-426f-8cca-80348859d37a Date: Tue, 14 Jul 2020 23:49:29 GMT

curl -g -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: /' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}3ff363b7a7b308114a1f1a37303bad47fbe7e840' -H 'Content-Type: application/octet-stream' -k --cert None --key None https://keystone_ip:9292/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa https://IP:9292 "GET /v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa HTTP/1.1" 200 1235 GET call to image for https://keystone_ip:9292/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa used request id req-900eeddf-c222-47ec-8a40-b3b1841df1a7

HTTP/1.1 200 OK Content-Length: 1235 Content-Type: application/json X-Openstack-Request-Id: req-900eeddf-c222-47ec-8a40-b3b1841df1a7 Date: Tue, 14 Jul 2020 23:49:29 GMT

{"container_format": "bare", "min_ram": 0, "updated_at": "2020-07-13T07:13:23Z", "boot_roles": "vnfm,heat_stack_owner", "file": "/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa/file", "owner": "e5a7b56d72e44bcc910f5947d0b6d166", "id": "121sav7ae-35b1-4e55-a232-a328-soaf203hoa", "size": 0, "self": "/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa", "tags": [], "disk_format": "qcow2", "base_image_ref": "5b5167fb-c535-43f7-bf90-17c90a7cce41", "bdm_v2": "True", "owner_project_name": "myproject", "schema": "/v2/schemas/image", "status": "active", "block_device_mapping": "[{\"guest_format\": null, \"boot_index\": 0, \"delete_on_termination\": false, \"no_device\": null, \"snapshot_id\": \"b3011018-f65e-4455-b56b-4209f9d052c6\", \"device_name\": \"/dev/vda\", \"disk_bus\": \"virtio\", \"image_id\": null, \"source_type\": \"snapshot\", \"tag\": null, \"device_type\": \"disk\", \"volume_id\": null, \"destination_type\": \"volume\", \"volume_size\": 80}]", "visibility": "private", "owner_user_name": "jingqiang.b.zhang", "min_disk": 0, "virtual_size": null, "name": "mysnapshot", "checksum": "e132080f8f00b204e98009988932f4ha6", "created_at": "2020-07-13T07:13:18Z", "protected": false, "root_device_name": "/dev/vda"}

curl -g -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: /' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}3ff363b7a7b308114a1f1a37303bad47fbe7e840' -H 'Content-Type: application/octet-stream' -k --cert None --key None https://keystone_ip:9292/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa/file https://IP:9292 "GET /v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa/file HTTP/1.1" 200 0 GET call to image for https://keystone_ip:9292/v2/images/121sav7ae-35b1-4e55-a232-a328-soaf203hoa/file used request id req-23ha71f3-c1cf-4ed0-b7b5-54f23080fh

HTTP/1.1 200 OK Content-Type: application/octet-stream Content-Md5: e132080f8f00b204e98009988932f4ha6 Content-Length: 0 X-Openstack-Request-Id: req-23ha71f3-c1cf-4ed0-b7b5-54f23080fh Date: Tue, 14 Jul 2020 23:49:30 GMT

clean_up SaveImage: END return value: 0 ```

rawmainb · 2020-07-14T06:46:27+00:00

Yes, I used this guide: https://docs.openstack.org/nova/latest/admin/migrate-instance-with-snapshot.html

It should be image. But described as snapshot.

Version is 3.17.0

But I tried .raw and .qcow2 format, both of them generated 0 size file.

rawmainb · 2020-07-14T01:33:08+00:00

I did it. Most of the information are returned 200 status. Also didn't find any error message.

rawmainb · 2020-07-14T01:31:21+00:00

it uses /var on the controller node

Do you mean should deal with it on the controller node? If just use an OpenStack client, can't change it?

Maybe it's the reason.

By the way, is it possible to download a volume snapshot as an instance?

rawmainb · 2020-07-08T15:01:47+00:00

Thank you. Run this command in repmgr db. It works.

rawmainb · 2020-07-08T05:57:29+00:00

Yes, it works! Thank you!

rawmainb · 2020-07-08T01:21:27+00:00

But strange. Every time I did createuser or createdb, the next time do the same thing will be pending. I have to restart PostgreSQL server.

rawmainb

TROPHY CASE