1b.app
Link copied -

Accident. System not working

Answers:

Works. Thank you for efficiency
27.02.2021, 14:54
Original comment available on version: ru

AP. Lie down for the second time.
27.02.2021, 16:22
Original comment available on version: ru

Good afternoon.
Judging by the logs on your VPS, there are problems either with the hardware part of the worm or with the database.
Contact your host and let them check your server.
Errors showed up Feb 27 03:07:35,
then Feb 27 03:09:44 DB errors appeared, at 03:14:22 errors stopped appearing.
Feb 27 14:32:43 DB errors appeared again.
Feb 27 14:49:30 the system restarted the database service and the errors were gone.
Feb 27 16:21:01 everything happened again Feb 27 16:43:06 the system restarted the database service
again errors and Feb 27 19:50:02 the system restarted the database service.
Feb 27 03:06:15 box systemd: Started Session 929500 of user root.
Feb 27 03:07:35 box kernel: INFO: task nginx:22158 blocked for more than 120 seconds.
Feb 27 03:07:35 box kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 27 03:07:35 box kernel: nginx D 0000000000000000 0 22158 27029 0x00000080
Feb 27 03:07:35 box kernel: ffff88000f127eb0 0000000000000282 ffff88003aefde20 ffff88000f127fd8
Feb 27 03:07:35 box kernel: ffff88000f127fd8 ffff88000f127fd8 ffff88003aefde20 ffff880002bfb6d8
Feb 27 03:07:35 box kernel: ffff880002bfb6dc ffff88003aefde20 00000000ffffffff ffff880002bfb6e0
Feb 27 03:07:35 box kernel: Call Trace:
Feb 27 03:07:35 box kernel: [<ffffffff8168d8e9>] schedule_preempt_disabled+0x29/0x70
Feb 27 03:07:35 box kernel: [<ffffffff8168b535>] __mutex_lock_slowpath+0xc5/0x1d0
Feb 27 03:07:35 box kernel: [<ffffffff8168a98f>] mutex_lock+0x1f/0x2f
Feb 27 03:07:35 box kernel: [<ffffffff811ff505>] SyS_write+0x65/0xe0
Feb 27 03:07:35 box kernel: [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
...
Feb 27 03:09:37 box kernel: Call Trace:
Feb 27 03:09:37 box kernel: [<ffffffff8168d8e9>] schedule_preempt_disabled+0x29/0x70
Feb 27 03:09:37 box kernel: [<ffffffff8168b535>] __mutex_lock_slowpath+0xc5/0x1d0
Feb 27 03:09:37 box kernel: [<ffffffff8168a98f>] mutex_lock+0x1f/0x2f
Feb 27 03:09:37 box kernel: [<ffffffff812ca326>] ima_file_check+0xa6/0x1b0
Feb 27 03:09:37 box kernel: [<ffffffff8120d76a>] do_last+0x58a/0x12a0
Feb 27 03:09:37 box kernel: [<ffffffff8120e542>] path_openat+0xc2/0x490
Feb 27 03:09:37 box kernel: [<ffffffff81180c2b>] ? unlock_page+0x2b/0x30
Feb 27 03:09:37 box kernel: [<ffffffff812106bb>] do_filp_open+0x4b/0xb0
Feb 27 03:09:37 box kernel: [<ffffffff8121d337>] ? __alloc_fd+0xa7/0x130
Feb 27 03:09:37 box kernel: [<ffffffff811fdbc3>] do_sys_open+0xf3/0x1f0
Feb 27 03:09:37 box kernel: [<ffffffff811fdcde>] SyS_open+0x1e/0x20
Feb 27 03:09:37 box kernel: [<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Feb 27 03:09:44 box mysqld: 2021-02-27 3:09:44 140608296249088 [Warning] InnoDB: A long semaphore wait:
Feb 27 03:09:44 box mysqld: --Thread 140608094070528 has waited at log0log.cc line 1090 for 241.00 seconds the semaphore:
Feb 27 03:09:44 box mysqld: Mutex at 0x7fe249ac2740, Mutex LOG_WRITE created log0log.cc:661, lock var 2
Feb 27 03:09:44 box mysqld: 2021-02-27 3:09:44 140608296249088 [Warning] InnoDB: A long semaphore wait:
Feb 27 03:09:44 box mysqld: --Thread 140608185423616 has waited at log0log.cc line 1090 for 241.00 seconds the semaphore:
Feb 27 03:09:44 box mysqld: Mutex at 0x7fe249ac2740, Mutex LOG_WRITE created log0log.cc:661, lock var 2
Feb 27 03:09:44 box mysqld: 2021-02-27 3:09:44 140608296249088 [Warning] InnoDB: A long semaphore wait:
...
Feb 27 03:14:22 box mysqld: 0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 1609.08 reads/s
Feb 27 03:14:22 box mysqld: ----------------------
Feb 27 03:14:22 box mysqld: END OF INNODB MONITOR OUTPUT
Feb 27 03:14:22 box mysqld: ===========================
Feb 27 03:14:22 box mysqld: InnoDB: ###### Diagnostic info printed to the standard error stream
...
Feb 27 14:32:43 box mysqld: 2021-02-27 14:32:43 140608296249088 [Warning] InnoDB: A long semaphore wait:
Feb 27 14:32:43 box mysqld: --Thread 140608170895104 has waited at log0log.cc line 1090 for 241.00 seconds the semaphore:
Feb 27 14:32:43 box mysqld: Mutex at 0x7fe249ac2740, Mutex LOG_WRITE created log0log.cc:661, lock var 2
Feb 27 14:32:43 box mysqld: 2021-02-27 14:32:43 140608296249088 [Note] InnoDB: A semaphore wait:
Feb 27 14:32:43 box mysqld: --Thread 140608094070528 has waited at log0log.cc line 1090 for 240.00 seconds the semaphore:
Feb 27 14:32:43 box mysqld: Mutex at 0x7fe249ac2740, Mutex LOG_WRITE created log0log.cc:661, lock var 2
Feb 27 14:32:43 box mysqld: 2021-02-27 14:32:43 140608296249088 [Note] InnoDB: A semaphore wait:
Feb 27 14:32:43 box mysqld: --Thread 140608085677824 has waited at log0log.cc line 1090 for 231.00 seconds the semaphore:
Feb 27 14:32:43 box mysqld: Mutex at 0x7fe249ac2740, Mutex LOG_WRITE created log0log.cc:661, lock var 2
Feb 27 14:32:43 box mysqld: 2021-02-27 14:32:43 140608296249088 [Note] InnoDB: A semaphore wait:
Feb 27 14:32:43 box mysqld: --Thread 140608172738304 has waited at dict0dict.cc line 1148 for 210.00 seconds the semaphore:
Feb 27 14:32:43 box mysqld: Mutex at 0x7fe249ad2b10, Mutex DICT_SYS created dict0dict.cc:1084, lock var 2
Feb 27 14:32:43 box mysqld: 2021-02-27 14:32:43 140608296249088 [Note] InnoDB: A semaphore wait:
Feb 27 14:32:43 box mysqld: --Thread 140608277927680 has waited at dict0dict.cc line 1148 for 231.00 seconds the semaphore:
...
Feb 27 14:49:24 box mysqld: Fatal signal 11 while backtracing
Feb 27 14:49:25 box systemd: mariadb.service: main process exited, code=killed, status=11/SEGV
Feb 27 14:49:25 box systemd: Unit mariadb.service entered failed state.
Feb 27 14:49:25 box systemd: mariadb.service failed.
Feb 27 14:49:27 box systemd: Started Session 930278 of user root.
Feb 27 14:49:28 box systemd: Removed slice user-0.slice.
Feb 27 14:49:29 box systemd: Created slice user-0.slice.
Feb 27 14:49:29 box systemd: Started Session 930280 of user root.
Feb 27 14:49:30 box systemd: Removed slice user-0.slice.
Feb 27 14:49:30 box systemd: mariadb.service holdoff time over, scheduling restart.
Feb 27 14:49:30 box systemd: Stopped MariaDB 10.2.31 database server.
Feb 27 14:49:30 box systemd: Starting MariaDB 10.2.31 database server...
...
Feb 27 16:21:01 box systemd: Started Session 930386 of user root.
Feb 27 16:21:48 box mysqld: 2021-02-27 16:21:48 139650684249856 [Warning] InnoDB: A long semaphore wait:
Feb 27 16:21:48 box mysqld: --Thread 139650589792000 has waited at log0log.cc line 1090 for 241.00 seconds the semaphore:
Feb 27 16:21:48 box mysqld: Mutex at 0x7f0354c75740, Mutex LOG_WRITE created log0log.cc:661, lock var 2
Feb 27 16:21:48 box mysqld: 2021-02-27 16:21:48 139650684249856 [Warning] InnoDB: A long semaphore wait:
Feb 27 16:21:48 box mysqld: --Thread 139650573006592 has waited at log0log.cc line 1090 for 241.00 seconds the semaphore:
...
Feb 27 16:43:01 box systemd: mariadb.service failed.
Feb 27 16:43:04 box systemd: Removed slice user-0.slice.
Feb 27 16:43:06 box systemd: mariadb.service holdoff time over, scheduling restart.
Feb 27 16:43:06 box systemd: Stopped MariaDB 10.2.31 database server.
Feb 27 16:43:06 box systemd: Starting MariaDB 10.2.31 database server...
...
Feb 27 17:05:01 box systemd: Started Session 930436 of user root.
Feb 27 17:05:03 box mysqld: 2021-02-27 17:05:03 140274492507904 [Warning] InnoDB: A long semaphore wait:
Feb 27 17:05:03 box mysqld: --Thread 140274500900608 has waited at log0log.cc line 1090 for 241.00 seconds the semaphore:
Feb 27 17:05:03 box mysqld: Mutex at 0x7f9499533740, Mutex LOG_WRITE created log0log.cc:661, lock var 2
Feb 27 17:05:03 box mysqld: 2021-02-27 17:05:03 140274492507904 [Warning] InnoDB: A long semaphore wait:
Feb 27 17:05:03 box mysqld: --Thread 140274484115200 has waited at srv0srv.cc line 2084 for 241.00 seconds the semaphore:
...
Feb 27 19:50:02 box systemd: Removed slice user-0.slice.
Feb 27 19:50:02 box systemd: mariadb.service holdoff time over, scheduling restart.
Feb 27 19:50:02 box systemd: Stopped MariaDB 10.2.31 database server.
Feb 27 19:50:02 box systemd: Starting MariaDB 10.2.31 database server...
27.02.2021, 20:42
Original comment available on version: ru

Thank you. Notified the technical support of the provider about failures.
27.02.2021, 20:55
Original comment available on version: ru

Got a reply from support:
With the server everything is garazd.
Here, imovirno, descriptions of your vipadok:
https://stackoverflow.com/questions/24860111/warning-a-long-semaphore-wait
27.02.2021, 21:54
Original comment available on version: ru

Good afternoon. We can't work, it's lying.
28.02.2021, 12:39
Original comment available on version: ru

Good afternoon.
If everything is in order with the hardware, then the problem is with the database. You need to restore the database.
But something tells me that not everything is as good with the server as the hoster said about it.
Now it is impossible to connect to the server, the response from the server is:
ssh_exchange_identification: Connection closed by remote host
Perhaps problems with the database arose due to problems with the disk subsystem, which is why I recommended
contact the host.
28.02.2021, 14:12
Original comment available on version: ru

Sent a second request to the hosting support
28.02.2021, 14:27
Original comment available on version: ru

Hoster support response:
Well, we don't have anything in the logs indicating problems with the disk subsystem.
But this possibility cannot be ruled out.
Now we will move you to another node, and on this we will run the tests.
It will take several reboots.
28.02.2021, 17:33
Original comment available on version: ru

Support sent a notification:
That's it, you are already on a new node.
If any problems arise - write.
28.02.2021, 18:39
Original comment available on version: ru

Notified all employees that the system is in test mode.
02.03.2021, 09:47
Original comment available on version: ru

Good afternoon.
As far as I can tell, after moving your VPS to a new node, the errors no longer appear (more than 40 hours have passed).
Apparently the problem was in the gland.
02.03.2021, 11:31
Original comment available on version: ru

Please join the conversation. If you have something to say - please write a comment. You will need a mobile phone and an SMS code for identification to enter. Log in and comment